RAG with PDFs

Why Use PDFs in RAG?

Companies store important knowledge in PDFs: product manuals, policies, reports, guides, invoices, research papers, and internal documentation. RAG lets an application retrieve the relevant parts instead of sending an entire PDF to the model.

The PDF-to-RAG Pipeline

Think of PDF RAG as two connected stages: ingestion prepares the files, while query time retrieves evidence for each question.

Step 1: Load a PDF

Python has several PDF libraries. For a simple text-based PDF, pypdf is a convenient teaching example.

from pypdf import PdfReader

reader = PdfReader("company-handbook.pdf")
print("Pages:", len(reader.pages))

The filename is an example. In a real application, the PDF may come from an upload, object storage, a document-management system, or another trusted source.

Step 2: Extract Text Page by Page

Keep page boundaries when possible. Page numbers become valuable metadata when you need to show where an answer came from.

from pypdf import PdfReader

reader = PdfReader("company-handbook.pdf")

pages = []
for page_number, page in enumerate(reader.pages, start=1):
    text = page.extract_text() or ""
    pages.append({
        "page": page_number,
        "text": text.strip(),
    })

for item in pages[:2]:
    print("Page", item["page"])
    print(item["text"][:500])

Step 3: Check What Was Extracted

Do not immediately embed extracted text. First inspect it. A PDF can look perfect to a human while producing poor machine-readable text.

Step 4: Clean the Extracted Text

Basic cleanup can remove repeated whitespace while keeping the actual content. Be careful not to remove meaningful formatting or identifiers.

import re

def clean_text(text):
    text = text.replace("\u00a0", " ")
    text = re.sub(r"[ \t]+", " ", text)
    text = re.sub(r"\n{3,}", "\n\n", text)
    return text.strip()

cleaned_pages = [
    {"page": item["page"], "text": clean_text(item["text"])}
    for item in pages
]

Cleaning should be evaluated against real PDFs. Aggressive cleanup can destroy useful structure.

Step 5: Chunk the PDF Text

Pages are not always good retrieval units. A page may contain several unrelated topics, while a section may continue across multiple pages. Chunk according to the information you want to retrieve.

def chunk_text(text, chunk_size=500):
    words = text.split()
    return [
        " ".join(words[i:i + chunk_size])
        for i in range(0, len(words), chunk_size)
    ]

chunks = []
for page in cleaned_pages:
    for chunk_index, chunk in enumerate(chunk_text(page["text"]), start=1):
        if chunk:
            chunks.append({
                "text": chunk,
                "page": page["page"],
                "chunk": chunk_index,
            })

print("Chunks:", len(chunks))

This fixed word window is only a teaching example. Structure-aware or recursive chunking may work better for real documents.

Step 6: Keep PDF Metadata

Every chunk should remain traceable to its source. At minimum, keep the document identifier and page number. You can also store section, title, language, version, tenant, or access information.

Step 7: Create Embeddings

Convert each chunk into an embedding so a user's question can be compared with the PDF knowledge semantically.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

texts = [item["text"] for item in chunks]
vectors = model.encode(texts)

print("Chunks:", len(texts))
print("Vector shape:", vectors.shape)

The model shown here is a teaching example. For production, evaluate an embedding model against your document language, query patterns, and retrieval benchmarks.

Step 8: Retrieve From the PDF

At query time, embed the question and rank PDF chunks by similarity. The top results become candidate evidence.

from numpy import dot
from numpy.linalg import norm

query = "How many days of annual leave are provided?"
query_vector = model.encode([query])[0]

def cosine(a, b):
    denominator = norm(a) * norm(b)
    return 0.0 if denominator == 0 else dot(a, b) / denominator

ranked = sorted(
    zip(chunks, vectors),
    key=lambda item: cosine(query_vector, item[1]),
    reverse=True,
)

top_k = ranked[:3]

for item, vector in top_k:
    print(f"Page {item['page']} | score={cosine(query_vector, vector):.3f}")
    print(item["text"][:300])

For larger collections, use a vector index or vector database instead of comparing every vector in Python.

Step 9: Build the RAG Context

Do not send the entire PDF to the LLM. Combine only the selected evidence and include enough source information to trace it.

context_parts = []

for item, vector in top_k:
    context_parts.append(
        f"[Source: company-handbook.pdf, page {item['page']}]\n"
        f"{item['text']}"
    )

context = "\n\n".join(context_parts)

prompt = f"""Answer the question using only the supplied PDF context.
If the context does not contain the answer, say that the information was not found.

Context:
{context}

Question:
{query}
"""

print(prompt)

Step 10: Generate the Answer

The final stage sends the question plus retrieved PDF evidence to an LLM. The LLM is responsible for explaining the evidence, not for replacing the retrieval system.

A production application can call its chosen LLM API here. The exact provider is independent of the PDF retrieval architecture.

PDF Tables Are Different

Tables are a common reason a PDF RAG system performs poorly. Extracted text may lose rows, columns, headers, or relationships between values.

Scanned PDFs and OCR

A scanned PDF may contain images of pages rather than selectable text. In that case, normal PDF text extraction can return little or no useful content.

OCR introduces its own errors, so important documents should be tested after extraction rather than assuming the OCR output is perfect.

When PDF RAG Works Well

PDF RAG vs Uploading the Whole PDF

For a tiny document, direct document input can be simpler. RAG becomes more useful when knowledge is large, distributed, changing, or needs retrieval controls.

Common PDF RAG Problems

Production Considerations

Improve PDF RAG Step by Step