What is RAG?

What Is RAG?

RAG stands for Retrieval-Augmented Generation. It is a pattern where an application first retrieves relevant information from an external knowledge source and then gives that information to a generative AI model as context for its answer.

Why Do We Need RAG?

An LLM is powerful, but a general model does not automatically know every private, current, or domain-specific document your application needs.

RAG does not magically make an LLM correct. It gives the model a better information source for the question.

The RAG Pipeline

In a real system, there are usually additional steps around this flow: document ingestion, chunking, embedding, indexing, filtering, reranking, prompt construction, citations, evaluation, and monitoring.

RAG Has Two Major Parts

Where Does the Knowledge Come From?

RAG can retrieve from many types of external knowledge. The source depends on the application.

Documents Become Chunks

Large documents are usually not placed into a vector index as one giant block. They are divided into smaller chunks so retrieval can find the specific parts that matter.

Good chunking is important because retrieval quality depends heavily on whether each stored chunk contains enough useful meaning without carrying too much unrelated text.

Embeddings and Retrieval

In semantic RAG systems, chunks are commonly converted into embeddings. An embedding represents text as a vector so that semantically related queries and documents can be compared.

A vector search can find the second text even when the query and document do not use exactly the same words.

What Happens During a User Query?

The exact implementation can differ. Some systems also use keyword search, metadata filters, hybrid search, reranking, query rewriting, or other retrieval strategies before generation.

Simple Python RAG Mental Model

The code below intentionally keeps retrieval and generation simple. It shows the core idea without requiring a specific vector database or LLM provider.

documents = [
    "You can reset your password from Account Settings.",
    "Shipping usually takes 3 to 5 business days.",
    "Invoices are available from the Billing page.",
]

query = "How do I reset my password?"

# Simplified retrieval for teaching purposes.
keywords = set(query.lower().replace("?", "").split())

retrieved = sorted(
    documents,
    key=lambda doc: len(keywords & set(doc.lower().split())),
    reverse=True,
)[:2]

context = "\n".join(retrieved)

prompt = f"""Answer the question using the context below.

Context:
{context}

Question:
{query}
"""

print(prompt)

In production, the retrieval step would normally use embeddings/search and the final prompt would be sent to an LLM API.

RAG Context Goes Into the Prompt

The retrieved chunks are not the answer by themselves. They become context that the generation model can use.

context = """
Password reset is available from Account Settings.
A password must contain at least 12 characters.
"""

question = "How do I reset my password?"

prompt = f"""
Use only the supplied context when answering.
If the context does not contain the answer, say that you do not have enough information.

Context:
{context}

Question:
{question}
"""

print(prompt)

This pattern is commonly called grounding: the application supplies relevant external information that the model can use when generating its response.

RAG vs Fine-Tuning

They solve different problems and can also be used together. RAG is primarily about supplying relevant external information at inference time.

RAG Does Not Mean the LLM Knows Everything

How RAG Fits With the Topics You Already Learned

What Makes a Good RAG System?

When Is RAG Useful?

Common RAG Mistakes

How to Evaluate RAG

Evaluate the retrieval stage and the final answer separately. A good LLM cannot compensate for consistently missing evidence.