RAG 3: useful context, hybrid search and reranking
What I would take from Contextual Retrieval to combine searches and rerank results, and what I would check before using it in production.
In this article
A sentence such as “the amount increased from the previous year” loses usefulness when separated from its entity, period and table. Anthropic’s contextual method adds a chunk-specific explanation before indexing. The idea is reasonable, but my first question would be whether those facts already exist in verifiable headings and metadata. I would not pay to generate what I can reliably extract.
Three representations for a fair comparison
I would compare original text, text with extracted context and text with generated context. For the third variant I would store the explanation, model, prompt, document version and review status separately. The generator would receive only the necessary content from the authorized document, not test-set answers. An invented explanation can improve an apparent match while reducing faithfulness.
search_text = verified_heading + "\n" + original_text
# Optional experimental variant:
contextual_search_text = generated_context + "\n" + original_text
# Citations always resolve to the original source.
cache_key = hash(document_version, chunk_id, model_version, prompt_hash)This is design pseudocode. The prefix helps search; it does not acquire documentary authority. When answering, the citation must resolve to the original passage and its location. If generated context asserts a period or entity unsupported by the source, I mark the enrichment invalid and retain the ability to retrieve without that prefix.
Combine signals without adding incompatible scales
I would retain BM25 as a baseline for identifiers and exact terms, alongside dense search for semantic similarity. I would compare each retriever separately before fusion. RRF combines ranks; adding scores requires calibration. Next I would deduplicate and test reranking over a bounded candidate set. Token budget and latency are part of the comparison.
I would not impose twenty passages as a universal rule. More candidates may improve retrieval but also introduce noise, incompatible versions and cost. I would tune these parameters on development data and freeze them before testing. For questions requiring several pieces of evidence, I would measure whether the necessary set arrived, not merely a related chunk.
What I would review in the notebook
The cookbook describes nine codebases and 248 queries. Its function named pass_at_n computes the fraction of reference chunks retrieved per query and then averages it: with multiple references, this is not binary success. It also compares exact text and retains queries with unresolved references in the denominator. In my implementation I would validate annotations first and use stable identifiers.
The article reports relative failure reductions tied to its evaluation, not answer accuracy. I would also not treat prices or model names from a page dated 2024 as a current contract. My adoption criterion for generated context would be useful local improvement over extracted context without degrading citations, permissions or the agreed cost budget.
Cookbook: implementation and evaluation