Ground model responses in selected private, domain-specific, or current evidence.
◷ Estimated reading time: 8 min
Why RAG Exists
Retrieval-augmented generation separates what the model learned during training from evidence selected at request time. It is useful when answers must depend on private documents, frequently changing information, or traceable sources.
Wrong evidence, too much context, unsupported answer
Retrieval Design Choices
Semantic search finds conceptually similar passages.
Keyword search is strong for identifiers, exact terms, and rare names.
Hybrid retrieval combines both signals.
Metadata filtering enforces tenant, date, region, or document-type boundaries.
Reranking spends more computation on a small candidate set to improve final ordering.
Long Context vs. RAG
Long context is useful when the full source set is small and every part may matter. RAG is preferable when the corpus is large, changes often, requires permissions, or needs source-level traceability. Many systems combine them: retrieval selects a compact set, then a long-context model reasons across it.
Separate Retrieval Failure From Generation Failure
Retrieval failure
The needed evidence was not found, filtered correctly, or ranked high enough.
Generation failure
The evidence was present, but the answer ignored, distorted, or overextended it.
Design rule
A good RAG system must be allowed to say "the available evidence is insufficient." Abstention is often more reliable than filling the gap from model memory.
Key Takeaways
RAG is an evidence-selection architecture, not simply "add a vector database."
Design and evaluate indexing and query-time pipelines separately.
Use citations and abstention to keep generation bounded by retrieved evidence.
Which step most directly improves the ordering of a small set of retrieved candidates?
An answer is wrong even though the correct passage appears in the supplied context. What failed?
When is metadata filtering especially important?
Which statement about long context and RAG is most accurate?