A Meta retrieval method, ported onto 4,195 of my own notes. The core claim holds. Two things in the reference design do not, and one of them is the prompt.
SiRA asks a language model to write extra search terms onto each document at ingest, then to expand the query the same way at search time. The idea is that both sides get pushed toward meeting in shared vocabulary, so a search for "car that drives itself" can reach a document that only ever says "autonomous vehicle."
It was published against clean academic benchmarks. I wanted to know whether it survives contact with a real personal corpus: markdown, YAML frontmatter, wikilinks, notes of wildly uneven length, and a topic mix nobody curated. So I ported the method rather than the code, and ran it against my own notes with my own questions.
The corpus grew six times over between two runs, from 719 documents to 4,195, by folding in 3,466 podcast summaries as deliberate noise. Same questions, same right answers, much bigger haystack. Everything below is the difference between those two runs.
The expected result was that a six times bigger pile would bury the answers. It did the opposite. Plain keyword search got better:
| corpus | Recall@10 | MRR |
|---|---|---|
| 719 docs | 0.825 | 0.702 |
| 4,195 docs | 0.875 | 0.675 |
The reason is mechanical. A word's weight in keyword search depends on how rare it is across the whole collection. Adding thousands of documents about completely unrelated topics made technical terms rarer by comparison, which turned them into sharper fingerprints. More hay made the needle shinier.
That reframed the whole experiment. The question was never "does the method survive a bigger haystack." The haystack was not the threat.
Not every generated term is worth indexing. A tag like "technology" sits on everything and helps you find nothing, so the method throws out any term that appears in more than 5% of the collection.
5% of 719 documents is 35. 5% of 4,195 is 209. The filter got six times more permissive at exactly the moment the collection got bigger.
Terms left of a line are kept. The 5% ratio rule at the larger scale waves through nearly everything, including terms that sit on 150+ documents and cannot distinguish between any of them.
The damage is easiest to see on terms whose real presence barely moved.
inference sits in 148 documents in the small corpus and 152 in
the large one. It became no more common. The ratio rule flips it from
rejected to accepted purely because the denominator grew underneath it.
Downstream, average tags per document nearly doubled, from 5.4 to 9.2. And the one configuration you would realistically deploy, tagging documents at ingest and then searching normally, went from slightly better than plain search to slightly worse than it.
An absolute cap solves it in one direction and breaks in the other: if the collection grows with more of the same, every count rises and eventually everything gets filtered. Ranking is stable under both. Reject a term if it sits among the K most common in the collection, and you are tracking the shape of the distribution rather than any count.
| filter | cap | tags/doc | doc-only R@10 | doc-only MRR | full R@10 | full MRR |
|---|---|---|---|---|---|---|
| 5% ratio | 209 | 9.2 | 0.850 | 0.715 | 0.975 | 0.781 |
| absolute | 35 | 5.4 | 0.900 | 0.711 | 0.950 | 0.777 |
| rank | 102 | 7.9 | 0.900 | 0.725 | 0.975 | 0.790 |
Ranking is best or tied best in every column. It recovers the deployable configuration without giving up anything in the full one.
The reference prompt is emphatic. It tells the model to produce terms that are not already in the document: "genuinely new vocabulary, not a rewording of what is already there." Bridging vocabulary is the entire stated mechanism.
Compliance turned out to be partial, which made it testable. Across 40,986 generated phrases, 44% contained no word from their source document and 56% reused at least one. I split the tags along that line and evaluated each half on its own.
| enrichment | full R@10 | full MRR | gold up / down | noise@10 |
|---|---|---|---|---|
| bridge, the intended kind | 0.875 | 0.743 | 6 / 4 | 0.175 |
| echo, the accidental kind | 0.950 | 0.799 | 9 / 0 | 0.152 |
| both together | 0.975 | 0.790 | 9 / 2 | 0.170 |
The accidental half is the strongest configuration measured in either run: the best ranking score, the only setup that never demotes a correct answer, and the only one where noise in the results drops below the untagged baseline. A paired bootstrap on the difference gives +0.056 MRR with a 95% interval of +0.017 to +0.105, better on 8 questions and worse on 1. The interval excludes zero, so it is a real effect at this sample size even though the recall column alone would be within noise.
A bridge phrase has no word in common with its document, so indexing it manufactures a match with nothing underneath. If the paraphrase drifts even slightly, the result is a false path straight to that document, and nothing in the phrase constrains the drift.
An echo phrase is mostly new vocabulary too. Only about a third of its words appear in the source. But that one shared word appears to be enough of a tether to keep the rest honest. There is likely a second effect compounding it: bridge phrases skew rarer, which means they carry more weight, which means each one swings a document's score harder. Evidence-free and high-leverage is a bad combination.
The honest limit: the echo tags are output the model produced while being told not to. That makes them a selected subset, possibly the terms it felt most strongly about. Whether the effect survives being asked for directly is untested, and it could come out worse.
The headline number is not the deployable number. Getting from 0.875 to 0.975 requires expanding the query at search time, which means a language model call sitting in the search path. Several seconds of latency on every query, for interactive search.
| configuration | R@10 | MRR | cost |
|---|---|---|---|
| plain keyword search | 0.875 | 0.675 | none |
| tag documents at ingest | 0.900 | 0.725 | one model call per document, once |
| tag plus expand queries | 0.975 | 0.790 | a model call on every search |
Strip out the search-time call and the honest gain is 2.5 points of recall and 5 points of ranking quality. Real, measured, and modest against a baseline that already works. On precisely worded questions the expansion is actively harmful: six correct answers demoted, none promoted.
On my own 17 real questions the picture is better, which is the result I trust most because I wrote those questions before seeing any output. The bigger corpus cost the baseline one answer, and enrichment won it back while lifting ranking quality from 0.785 to 0.833.
So the method works, the reference implementation has a scaling bug in its filter, and its prompt is arguing against its own best mechanism. Whether any of that is worth deploying depends entirely on whether you can afford a model call in the search path. I cannot, so for now this stays a finding rather than a feature.