What is Retrieval?
Learn how a Generative AI system finds the most relevant information before an LLM generates an answer.
By the end of this lesson, you will be able to:
- Explain retrieval and why it is important in RAG systems.
- Describe how a user query becomes a retrieval request.
- Understand embeddings, similarity search, top-k, and metadata filtering.
- See the difference between retrieving evidence and generating an answer.
- Build a simple Python retrieval mental model.
- Evaluate whether a retriever returns useful evidence for real questions.
What Is Retrieval?
Retrieval is the process of finding relevant information from a collection of stored data in response to a query.
In a RAG system, retrieval happens before generation. The system searches documents, chunks, or other knowledge sources and selects useful pieces of evidence for the language model.
User asks: “How many days do I have to return a damaged product?” Retrieval finds the relevant returns-policy chunk.
User asks: “How do I reset an API key?” Retrieval finds the documentation section that describes the reset process.
Where Retrieval Fits in RAG
A typical RAG workflow separates knowledge retrieval from answer generation.
The retriever does not need to write the final answer. Its job is to return useful context with enough relevance and coverage for the next stage.
The Retrieval Query
The retrieval process starts with a query. This can be the user's original question, a rewritten question, or another search representation created by the application.
Natural language
“Can I return a damaged item after 20 days?”
Search intent
Find policy content about damaged-item return deadlines and conditions.
How Semantic Retrieval Finds Meaning
Modern RAG systems often convert the query and stored chunks into embeddings. Each embedding is a numerical vector representing aspects of the content's meaning.
The retriever then compares the query vector with stored vectors using a similarity or distance measure. Chunks with stronger matches become retrieval candidates.
Retrieval From a Vector Index
After documents are chunked and embedded, their vectors can be stored in a vector index or vector database. At query time, the query embedding is compared against those stored vectors.
Retrieval is therefore connected to earlier lessons: chunking determines the units being searched, embeddings represent them, and the vector index makes similarity search practical.
What Does Top-k Mean?
Top-k means returning the best k retrieval candidates according to the search method.
One result
Focused, but risky if the first result misses important evidence.
Three results
Provides more coverage while keeping the context relatively focused.
Ten results
Can improve recall, but may introduce noise and consume more context.
Metadata Filtering
Similarity is not always enough. Metadata can restrict retrieval to documents that satisfy business or application rules.
Search only documents where product = "Model X" before ranking similar chunks.
Search only policy documents where year = 2026 when the current policy matters.
This combination is powerful: semantic similarity finds relevant meaning, while metadata filters enforce known constraints.
Simple Python Retrieval Mental Model
You can understand the basic idea without a vector database first. Imagine that each stored item already has a similarity score against the query.
results = [
{"text": "Returns are accepted within 30 days.", "score": 0.92},
{"text": "Warranty coverage lasts one year.", "score": 0.61},
{"text": "Shipping usually takes 3 days.", "score": 0.34},
]
top_k = sorted(results, key=lambda item: item["score"], reverse=True)[:2]
for item in top_k:
print(item["score"], item["text"])This example skips the embedding and indexing steps. It shows the core retrieval idea: rank candidates by relevance and keep the strongest matches.
Retrieval Is Not Generation
- Finds evidence
- Ranks candidates
- Applies filters
- Returns context
- Reads retrieved context
- Follows instructions
- Produces natural language
- Forms the final response
A retriever can return excellent evidence while the generator still produces a poor answer. Conversely, a strong language model cannot reliably answer from private or changing information if the relevant evidence was never retrieved.
Retrieval and RAG Context
In RAG, retrieved chunks are normally assembled into a context that is passed to the language model.
Exact Search vs Semantic Retrieval
Can miss a relevant document when the user and document use different words for the same concept.
Can match related meaning even when the wording is different, using embeddings and similarity.
Neither approach is automatically best for every workload. Exact terms, identifiers, product codes, and names can be especially important, while semantic retrieval is valuable for natural-language questions.
How to Evaluate Retrieval
Do not evaluate retrieval only by looking at a few impressive examples. Create realistic questions and define what evidence should be retrieved.
Measure how often the expected evidence appears in the retrieved set.
Check whether relevant evidence appears near the top of the results.
Verify that the retrieved chunk contains enough information to answer.
Finally check whether the retrieved evidence supports a correct answer.
Common Retrieval Mistakes
Large result sets can add noise and consume the model's context budget.
Top-1 may miss a second chunk needed to complete the answer.
Similarity alone cannot enforce every business or time constraint.
A retriever cannot recover context that was destroyed during chunking.
Retrieval quality must be measured across a representative question set.
A high similarity score means a strong match, not guaranteed factual correctness.
What Makes a Good Retriever?
A good retriever consistently brings the right evidence near the top while keeping irrelevant content under control.
Practice: Build a Tiny Retriever
Create five short text chunks and assign each a simple relevance score for a sample question. Sort the results and inspect the top three. Then ask whether the top results contain enough evidence to answer the question.
Inspect the evidence
Try a second question and see whether the same ranking still makes sense. Think about what could go wrong when scores are based on real embeddings.
Quick Quiz
1. What is retrieval?
Finding relevant information from a stored collection in response to a query.
2. What does top-k mean?
Returning the best k candidates according to the retrieval ranking.
3. Why use metadata filters?
To enforce known constraints such as product, source, tenant, language, or date before or during retrieval.
4. Does a high similarity score guarantee a correct answer?
No. It indicates a strong match according to the retrieval measure, but the evidence still needs to be relevant, sufficient, and accurate.
30-Second Recap
Find useful evidence for a query.
Order candidates by relevance.
Use metadata when constraints matter.
Give selected context to the LLM.
Key Takeaways
- Retrieval finds relevant information before generation in a RAG workflow.
- Semantic retrieval commonly uses embeddings and similarity search.
- Top-k controls how many candidate results are returned.
- Metadata filters can enforce constraints that similarity alone cannot express.
- Retrieval quality depends on chunking, embeddings, indexing, queries, filters, and ranking.
- Evaluate retrieval with realistic questions and expected evidence, not intuition alone.