PDF Question Answering

What Are We Building?

We are building a small RAG application for asking questions about a PDF. The user uploads a document, asks a question, and receives an answer grounded in relevant passages from that document.

Understand the End-to-End Architecture

Keep ingestion and question answering as two separate flows. The PDF is processed once during ingestion; questions reuse the prepared index.

Separate Ingestion from Querying

A common design mistake is re-reading and embedding the PDF every time a user asks a question. Instead, create the index once and reuse it.

Extract Text from a PDF

PDFs are layout-oriented files, so extraction quality matters. Start with a library that can return text page by page, then preserve page metadata for citations.

from pathlib import Path


def extract_pages(pdf_path: str) -> list[dict]:
    # Replace this demo reader with your PDF library.
    text = Path(pdf_path).read_text(encoding="utf-8")
    return [{"page": 1, "text": text}]

pages = extract_pages("manual.txt")
print(pages[0]["page"])

For a real PDF, use a PDF parser such as PyMuPDF or another suitable document extraction library.

Preserve Page Metadata

Do not throw away where a passage came from. Page numbers, document IDs, section names, and chunk IDs make answers easier to verify and debug.

Clean Extracted Text

Extraction can produce repeated headers, footers, broken whitespace, and line breaks. Clean obvious noise without destroying meaningful structure.

import re


def clean_text(text: str) -> str:
    text = re.sub(r"[ \\t]+", " ", text)
    text = re.sub(r"\\n{3,}", "\\n\\n", text)
    return text.strip()

cleaned = clean_text("Page title   \\n\\n\\n\\n  Useful content")
print(cleaned)

Choose a Chunking Strategy

Chunking determines the units that retrieval can return. For PDFs, page-aware recursive or semantic chunking is often more useful than blindly cutting every N characters.

Create Chunks with Metadata

Each chunk should carry both its text and source information. This lets you return citations and filter results later.

def make_chunks(pages: list[dict], size: int = 800) -> list[dict]:
    chunks = []
    for page in pages:
        text = page["text"]
        for start in range(0, len(text), size):
            chunks.append({
                "chunk_id": f"p{page['page']}-{start}",
                "page": page["page"],
                "text": text[start:start + size],
            })
    return chunks

chunks = make_chunks(pages)
print(len(chunks))

Generate Embeddings

Convert each chunk into a vector representation so semantically related questions and passages can be compared.

Store Vectors in an Index

A vector database or local vector index lets you search many chunks efficiently. Store the chunk text and metadata alongside the vector or in a linked document store.

Embed the User's Question

The query must be represented in the same embedding space as the indexed chunks.

def embed(text: str) -> list[float]:
    # Demo placeholder for an embedding model.
    return [float(len(text)), 0.5]

question = "What is the refund policy?"
query_vector = embed(question)
print(query_vector)

Retrieve the Most Relevant Chunks

Use similarity search to get a candidate set, such as the top 5–10 chunks. Retrieval quality is often the biggest factor in whether a RAG answer is grounded.

Build a Grounded Prompt

Tell the model that the supplied context is the source of truth. Make the citation metadata visible to the model and instruct it not to invent unsupported details.

Assemble Context in Python

Keep retrieved chunks structured until the final prompt is built.

def build_context(results: list[dict]) -> str:
    parts = []
    for item in results:
        parts.append(
            f"[Page {item['page']} | {item['chunk_id']}]\\n{item['text']}"
        )
    return "\\n\\n".join(parts)

context = build_context(chunks[:2])
print(context)

Generate the Answer

The LLM receives the question plus retrieved evidence. Keep temperature and output length appropriate for factual document QA.

Return Sources with the Answer

A good PDF QA interface should show the evidence used to answer. This makes the system more trustworthy and gives the user a way to verify the response.

• Answer text
• Document name
• Page number
• Relevant excerpt
• Confidence or retrieval score when appropriate
• Link to the source page when your viewer supports it

Handle Questions the PDF Cannot Answer

Retrieval can return weak matches even when the answer is absent. Add a relevance threshold or evaluation step so the application can refuse to guess.

def should_answer(results: list[dict], threshold: float = 0.72) -> bool:
    if not results:
        return False
    return results[0].get("score", 0.0) >= threshold

results = [{"score": 0.81}]
print(should_answer(results))

Support Multiple PDFs

When users upload several documents, add a document ID to every chunk. Then you can search across all documents or filter to one selected document.

PDFs Are Not Always Text

Scanned PDFs may contain images instead of machine-readable text. Tables, diagrams, columns, and unusual layouts can also reduce extraction quality.

Improve Retrieval Quality

If answers are weak, do not immediately change the LLM. First inspect whether the correct evidence was retrieved.

• Improve chunk boundaries
• Adjust chunk overlap
• Change embedding model
• Increase candidate Top-K
• Add metadata filtering
• Use reranking

Control Context Size and Cost

Sending ten large chunks for every question can increase latency and token cost. Retrieve a reasonable candidate set, rerank when needed, and send only the strongest evidence.

Add Security and Privacy Controls

Uploaded documents can contain sensitive information. Treat the PDF pipeline as an application boundary, not just a demo script.

• Validate file type and size
• Scan uploads before processing
• Isolate users and tenants
• Protect stored documents and vectors
• Limit who can query each document
• Do not expose secrets in prompts or logs

Evaluate the PDF QA System

Create questions with known answers and source pages. Evaluate both retrieval and generation separately.

Mini Project: Build a PDF Q&A Assistant

Build a small application that accepts a PDF and answers questions using retrieved passages.