Half of My Hybrid Search Was Silently Off

TL;DR — I pointed my own retrieval engine at my own blog and it ranked the wrong post first. Not dirty data, not bad embeddings: one word of Rust. Search terms were joined with a space, which SQLite's full-text search reads as AND, so every natural-language question silently zeroed out the keyword half of "hybrid" search and left it running on embeddings alone. Swapping the space for OR moved the right answer from #3 to #1.

Takeaway: retrieval is an engineering surface with its own silent failures — so don't reach for it until your context genuinely won't fit in the model's window. Cache-and-dump (paste it all in, let caching pay for it) is the boring default that wins until it physically can't.

I asked my own search engine for the one post I've written about saving tokens with a Rust CLI. I've written exactly one. It came back third.

❯ ctx search "the post about saving tokens with a rust cli" --mode hybrid --explain
Search: mode=hybrid, alpha=0.60, candidates: 0 keyword + 80 vector

1. [0.60] filesystem:blog / rust-memory-cheatsheet.md
    scoring: keyword=0.000  semantic=1.000  → hybrid=0.600
2. [0.56] filesystem:blog / nes1.md
    scoring: keyword=0.000  semantic=0.927  → hybrid=0.556
3. [0.45] filesystem:blog / rtk-token-killer.md
    scoring: keyword=0.000  semantic=0.745  → hybrid=0.447

Ahead of the right answer: a Rust memory cheatsheet, and an emulator for the NES. The post it should have handed me — "How I Cut My Claude Code Token Usage by 90% with RTK", tagged tokens, rust, cli, transparently about saving tokens with a Rust CLI — lost to two posts whose main qualification is that they also say "Rust" a lot.

Nothing here is dirty. The corpus is 40 blog posts I wrote and proofread. The right document is in the index. The query is a plain English question. And Context Harness — the retrieval engine I built and have been writing this series about — confidently ranked a cheatsheet about smart pointers above it.

Last post I promised this one would be "what retrieval actually buys you, and the specific moments it's slop in, slop out." I want to correct that framing before I use it, because "slop in, slop out" is garbage-in-garbage-out, and GIGO is the most-written and least-interesting take in all of RAG: clean your chunks, tune your overlap, and so on. My chunks are clean. What bit me was something else, and — I'll spoil the ending — it turned out to be one word of my own code. So this is two posts in one: the bug and the fix, which is the honest shape of building anything; and then the larger question the bug throws into relief, which is whether you should be reaching for retrieval at all.

The take I'm not writing

The failure above is not dirty data. It's structural, and it hides inside the word everyone reaches for when they want retrieval to sound trustworthy: hybrid.

The pitch for hybrid search is genuinely good, which is why everyone repeats it. Keyword search — BM25 over an inverted index — nails exact terms and whiffs on paraphrase: it'll find "BM25" and miss "that ranking function with term-frequency saturation." Semantic search — cosine similarity over embeddings — is the mirror image: catches the paraphrase, misses the exact string. So you run both and blend the scores. Keyword covers semantic's blind spot; semantic covers keyword's. A safety net with two independent ropes.

Context Harness does exactly this, on purpose. Keyword candidates come from SQLite's FTS5 bm25(). Vector candidates come from brute-force cosine similarity over every chunk — no approximate index, just the dot products. Each side is min-max normalized per query, then blended:

hybrid = (1 - alpha) * keyword_norm + alpha * semantic_norm

with alpha = 0.6, leaning slightly toward semantics. That's the whole scoring model. Two ropes, one knot. So why did a post that's about saving tokens with a Rust CLI lose to a memory cheatsheet?

Read the first line of the output

Search: mode=hybrid, alpha=0.60, candidates: 0 keyword + 80 vector

Zero keyword candidates. One rope wasn't there. "Hybrid" quietly degraded to pure semantic search, and pure semantics decided "rust cli tokens" was closest to a cheatsheet about Rust memory. I only know one rope was missing because I passed --explain; without it I'd have gotten a clean-looking ranked list and no hint that half the machinery had switched off.

Why zero? Watch it flip on a single word:

❯ ctx search "tokens rust cli" --mode keyword
1. [1.00] filesystem:blog / rtk-token-killer.md

❯ ctx search "saving tokens rust cli" --mode keyword
(no results)

tokens rust cli finds the right post instantly, perfect score. Add saving — a word that isn't in that post — and the entire result set collapses to nothing. That is the behavior of an AND: every term required, one miss and you get zero. And a natural-language question is a pile of words your target document mostly doesn't contain (the, post, about, saving, with, a), so under AND the keyword rope goes slack on exactly the queries a human or a model actually types. It only holds for keyword-salad input that nobody writes at a chat box.

I didn't design an AND. So I opened the file.

One word

The query text becomes an FTS5 MATCH string in one small function (sqlite_store.rs):

fn fts_query_from_user_text(query: &str) -> String {
    query
        .split(|c: char| !(c.is_alphanumeric() || c == '_'))
        .filter(|term| !term.is_empty())
        .collect::<Vec<_>>()
        .join(" ")          // ← the bug
}

It splits the query into words and joins them with a space. And in FTS5, a space between terms is an implicit AND. So the post about saving tokens with a rust cli becomes a query that means every one of those words must appear in a single document — which nothing does. The keyword arm wasn't broken. It was doing precisely what I told it to, and what I told it was wrong.

You can watch the fix work before changing any Rust, straight against the FTS index — space (AND) versus OR:

-- implicit AND (space-joined): the current behavior
sqlite> SELECT bm25(chunks_fts), title FROM chunks_fts ...
        WHERE chunks_fts MATCH 'the post about saving tokens with a rust cli' ...;
-- (no rows)

-- OR-joined
sqlite> ... WHERE chunks_fts MATCH 'the OR post OR about OR saving OR tokens OR ... OR cli' ...;
-11.43  rtk-token-killer.md
 -9.01  context-harness-rag-rust.md
 -7.65  rtk-token-killer.md

OR lets a document match on any term, and BM25 does the rest: common words like the and with appear everywhere, so their inverse-document-frequency weight is near zero and they contribute almost nothing; rare, load-bearing words like tokens and rust and cli drive the score. The right post wins on the terms that matter. So the fix is one word:

-        .join(" ")
+        .join(" OR ")

The after

Rebuild ctx, re-run the exact query that started this — same corpus, same index, nothing re-embedded, only the query construction changed:

❯ ctx search "the post about saving tokens with a rust cli" --mode hybrid --explain
Search: mode=hybrid, alpha=0.60, candidates: 80 keyword + 80 vector

1. [0.85] filesystem:blog / rtk-token-killer.md
    scoring: keyword=1.000  semantic=0.745  → hybrid=0.847
2. [0.69] filesystem:blog / rust-memory-cheatsheet.md
    scoring: keyword=0.224  semantic=1.000  → hybrid=0.690
3. [0.67] filesystem:blog / nes1.md
    scoring: keyword=0.364  semantic=0.878  → hybrid=0.673

candidates: 80 keyword + 80 vector — both ropes taut. The token-killer post goes from third to first; the cheatsheet and the emulator, which only ever had the semantic rope, drop below it.

That's the entire fix, and it's the whole reason to instrument retrieval in the first place: the --explain line is what turned "the results feel off" into "one arm reports zero candidates," which is a bug you can find in a file instead of a vibe you argue about.

It wasn't a one-off, either. Earlier I'd asked a four-repo index — 446 documents, roughly 700,000 tokens — how its own hybrid scoring normalizes, and the document literally titled hybrid-scoring-with-min-max-normalization came in second, beaten by fifteen thousandths of a point by a vaguer doc that caught the one stray keyword hit. After the fix:

❯ ctx search "how does hybrid scoring normalize keyword and semantic scores" --mode hybrid --explain
1. [1.00] filesystem:ch / 0005-hybrid-scoring-with-min-max-normalization.md
    scoring: keyword=1.000  semantic=1.000  → hybrid=1.000

First, perfect score. The engine can now find the spec that describes the engine. And the query that already worked — a keyword-rich set up remote build machines for nix, which was pulling its right answer at a clean 1.000 before — still sits at 1.000 after, with the cross-repo runbook it surfaced from a different repo actually scoring higher than before. On the three queries that defined the problem, the right answer is now first, and the control didn't regress.

I'm being careful with that sentence, because OR is not free — it broadens the candidate pool, so somewhere it has to cost me. So I ran a bigger check: fifteen queries by hand across both corpora, each with the answer I expected written down first. Twelve came back identical, two improved, and one regressed — how do I run a database in the browser used to put my database-in-the-browser post first, and now ranks a closely related SurrealDB-caching post there instead, with the on-the-nose post bumped to second. OR pulled in a neighbor and the blend preferred it: a real, if small, tax, landing exactly where I'd have guessed — a keyword-shaped query where AND's precision was quietly doing work. Better than three cherry-picked wins. Still fifteen queries I chose, graded against answers I decided were right, which is not a benchmark — it's marking my own homework, and it's the next post's problem. For now the honest claim is narrow and true: a one-word bug was zeroing out half of hybrid search on every natural-language query, and now it isn't.

The bug is the argument

Here's the part that outlasts the fix. A single word — a space where an OR belonged — silently disabled one of the two retrieval strategies I built the whole engine around, on exactly the inputs it would see in real use, and it produced confident, plausible, cited wrong answers the entire time. No crash. No error. A ranked list with scores, every one of them a lie of omission. I'd have kept trusting it if I hadn't gone looking.

That is the true cost of retrieval, and it's why the interesting question isn't "how do I make RAG better," it's "should I be running RAG here at all." Because look at what the whole episode was for: those 40 blog posts are about 60,000 tokens. They fit in a context window with room to spare. The correct thing to do with a corpus that small is not to retrieve over it — it's to paste every post into the prompt, let prompt caching make the repeat cost trivial, and let the model read all of it. Do that and this bug cannot happen to you, because there's no ranking step to get wrong. There's no #1 to incorrectly promote, because there is no list. I introduced an entire class of silent failure to solve a problem — "which twelve chunks?" — that, at this scale, I did not have.

Cache-and-dump is the monolith of context: unglamorous, and correct right up until it physically won't fit in the window. Retrieval is the service you extract when the monolith overflows — and, exactly like extracting a microservice, the moment you do it you sign up for an operational surface the monolith never had. Query construction. FTS semantics. Score normalization. A one-word bug that halves your quality without telling you. You don't take that on because it's elegant. You take it on because you were forced to. And the reasons that force you are specific — not "it's cheaper" (with caching, dumping is cheap; the invoice going up isn't the work getting better), but capability:

1. Scale past the window — a corpus you genuinely can't fit, like those four repos.
2. Relevance you can't predict — the one document out of hundreds you didn't know you'd need until the question existed.
3. A corpus that changes — so you don't re-shovel and re-cache everything on every edit.
4. Attention rot — even when it all fits, more context is not free accuracy; a model handed 700K tokens attends worse than one handed the right 5K.

That cross-repo runbook the fixed engine surfaced — a doc from one repository answering a question I'd only documented in another, neither aware the other exists — is retrieval earning its keep, and it earned it because reason #1 was real: 700K tokens is not going in a window. On the blog, none of the four apply, so retrieval had no job, and inventing one for it is how I ended up ranking a smart-pointer cheatsheet above the post I asked for.

This is the same lesson as last post, one layer down. There I'd chopped a solo project into 78 documents and six agents because decomposition felt like rigor, and called it microservices-on-myself. Retrieval on a corpus that fits in the window is the identical mistake: paying the coordination cost of a distributed system to solve a problem a monolith already had handled. Extract the service when you're forced. Not before.

Distrust the retriever — enough to instrument it

There's a discipline I lean on hard at work; I'll keep it generic, but it travels: treat the model as an advisor to be verified, not an authority to be trusted. You don't take its confident output at face value — you build the system so the output gets checked. Almost nobody extends that same suspicion to the retriever. Retrieved context gets a pass. It feels like ground truth: it came from your own documents, it has a similarity score, it's the thing you point to when someone asks whether the model made it up. But rust-memory-cheatsheet came back at 0.60 for a question about token savings, and that number was never a measure of whether it was right — it was a measure of embedding proximity with one of two ropes cut, which is a very different thing wearing the same costume. The model downstream can't tell the difference. It receives the top chunk as grounding and does what it does with grounding: builds a confident, well-cited answer on top of it. On a bad ranking, retrieval doesn't remove the confabulation risk — it supplies the confabulation with a footnote.

So distrust it. But the useful form of distrust isn't a shrug about how retrieval is treacherous; it's --explain. Instrument the thing so that "this feels wrong" becomes "the keyword arm returned zero candidates," which is a bug with a location and a fix. Suspicion that ends in humility is a pose. Suspicion that ends in a diff is engineering.

Where this goes

Context Harness got one bug better this afternoon, in public, and I'll take it — but the honest scope of what I proved is: on three queries that used to fail, the right answer is now first. The reranker I keep deferring would catch a different, harder class of miss; OR-joining is not the last word on query construction; and I still can't tell you, with a number, whether the engine is good — only that it's better on the cases I looked at, which is exactly how you fool yourself.

Which is the next problem, and a harder one than a one-word bug: I've spent this whole post saying "first" and "better" and "wrong" as if I can measure any of it. I mostly can't — not the way a benchmark would — and the gap between what I can measure and what actually matters is where the next post lives.

— Parker Jones