Agent Retrieval
RAG Patterns
The retrieval pipeline from chunking through reranking to agentic and graph-augmented RAG - condensed current practice
Last verified 2026-10-29
comes in many shapes. Treat each named variant below as a design move; most production systems combine several: hybrid retrieval, , contextual chunks, and .
Production default pipeline
- Ingest with stable document IDs, metadata, permissions, and source hashes.
- Chunk semantically (512–1,024 , 10–30% overlap is a reasonable starting default); add contextual chunk descriptions where ambiguity is common.
- Retrieve with hybrid search: dense plus a sparse ( or full-text) index, fused with Reciprocal Rank Fusion or a database-native hybrid query.
- Rerank the top 50–100 candidates down to 5–10. Rerankers are separate models; Cohere Rerank 5 (released 2026-10-21) is one dated example, with
rerank-v5.0-proat $0.05 per 1M tokens andrerank-v5.0-fastat $0.02, per the announcement. It is an option, not a required component. - Enforce metadata and authorization filters during retrieval.
- Generate with citations back to source chunks.
- Evaluate recall, ranking quality, faithfulness, latency, and cost - see Retrieval Evals.
Keep chunk metadata (document_id, source, section, tenant_id, updated_at, content_hash) with every indexed unit - chunking without produces unattributable, unfilterable results.
Pattern taxonomy
| Pattern | Use when | Avoid when |
|---|---|---|
| Naive (embed, retrieve top-k, generate) | Prototypes, small clean corpora | Production - fails on exact identifiers and multi-hop questions |
| Hybrid (BM25 + dense + fusion) | Corpus has product names, codes, or IDs alongside prose | Rarely - this is the default production stance |
| Reranked | Retrieved context is noisy | Ingestion or chunking is broken - reranking cannot recover a missing passage |
| Contextual (Anthropic's Contextual Retrieval pattern: prepend a chunk-context summary before indexing) | Chunks are ambiguous without surrounding document context | Index-time cost or staleness of generated context is unacceptable |
| Parent-child | Small chunks rank well but generation needs a full section | - |
| Hierarchical / RAPTOR | Long documents need both local facts and document-level synthesis | Documents change frequently, or exact-source traceability matters more than abstraction |
| HyDE / multi-query / RAG-Fusion | Queries are short, vague, or phrased differently from source docs | Latency and model cost outweigh the gain on simple lookups |
| GraphRAG | Relationships are first-class, multi-hop, and worth maintaining | Entity resolution is unreliable, or hybrid retrieval already answers most questions - see Knowledge Graphs |
| Agentic RAG (agent plans subqueries, chooses tools, iterates) | Multi-step questions spanning documents, tables, and | Simple FAQ-style lookups - adds latency, cost, and failure surface |
| Self-RAG / Corrective RAG | Factuality matters and retrieval quality varies by query | You cannot afford extra model calls per query |
| Multimodal RAG (ColPali-style late interaction) | Source truth is in charts, tables, scans, or slides | OCR-based chunking already answers most queries |
| Structured/tool-backed (SQL, , tools) | The answer depends on live state, joins, or permissions | Never; the retrieval layer here is a live query against a system of record |
Hybrid retrieval and fusion
function reciprocalRankFusion(resultLists: SearchResult[][], limit = 50, k = 60) {
const scores = new Map<string, { score: number; result: SearchResult }>();
for (const results of resultLists) {
results.forEach((result, index) => {
const existing = scores.get(result.id);
scores.set(result.id, {
score: (existing?.score ?? 0) + 1 / (k + index + 1),
result,
});
});
}
return [...scores.values()].sort((a, b) => b.score - a.score).slice(0, limit);
}Agentic retrieval as a bounded escalation
Use agentic retrieval (query planning, multi-hop iteration, tool selection) for multi-hop questions, schema-plus-vector queries, or answer validation. Put hard bounds around it: maximum sub-queries, maximum tool calls, a token and cost budget per request, required metadata filters, and IDs for replay.
Anti-patterns
- Naming the architecture after the newest acronym instead of the failure mode it fixes.
- Adding graph retrieval before entity resolution is reliable.
- Using agentic RAG for single-hop lookups.
- Expanding queries without reranking.
- Measuring answer quality without measuring retrieval quality.