Your RAG Cites Sources Like a Student at 2am
There is a footnote. It looks official. Nobody actually checked whether the passage says what the answer claims. In a term paper that costs you a grade. In an FDA submission it costs a lot more, and "it sounded right" is not a defense anyone accepts.
Here is the trap. Plain keyword search gets fooled every single time by a dead document. A rejected draft that happens to repeat your exact query words will outrank the real, current passage, because it looks like a better match and has no idea it was supposed to stay buried.
citebench is a small, readable RAG pipeline (BM25, dense fusion, reranking) built for one purpose: to measure, in a number, exactly how much reranking rescues you from that failure.
Three Stages, Each One Measurable
It is built in layers so you can see what each one is worth.
• Lexical: a pure-stdlib Okapi BM25 index. The keyword baseline that gets fooled.
• Dense fusion: a hashing dense index blended with BM25 scores, alpha-weighted.
• Reranking: a cross-encoder-style scorer that punishes superseded and draft sources and rewards real phrase overlap over cheap keyword stuffing.
• Validation: a hard confidence floor. If the top citation cannot clear it, the pipeline says "no confident citation" instead of confidently pointing at garbage.
A Benchmark That Fights Back
A benchmark is worthless if the corpus is polite, so I built the traps by hand. A rejected draft that keyword-stuffs the exact query. A superseded manual. The zombie documents that refuse to stay dead in any real archive. Take those out and BM25 looks brilliant and the whole exercise means nothing.
The other rule was zero dependencies and zero API keys, so anyone can clone it and reproduce the number in under a minute without renting an embedding API to fact-check my claim.
The Number
On the adversarial set, keyword retrieval sits at 62% citation precision whether or not you bolt dense fusion on top. Add reranking and it jumps to 88%, and every point of that gain traces directly to the reranker throwing out the dead and keyword-stuffed sources.
7 tests green across Python 3.9 through 3.13, with the example and the full benchmark running on every push.