Evaluation

What Is GenAI Evaluation?

Evaluation is the systematic process of measuring an AI application's behavior against defined expectations. It helps you decide whether a prompt, model, retriever, tool workflow, or full application is good enough for users.

Why Evaluation Matters

Production GenAI systems can change when you update a model, prompt, chunking strategy, retrieval configuration, tool, or safety policy.

• Catch regressions: A new change may improve one task and break another.
• Compare models: Measure quality before choosing a cheaper or faster model.
• Improve RAG: Separate retrieval problems from generation problems.
• Measure agents: Check whether the agent selected the right tools and completed the task.
• Control risk: Test unsafe, incorrect, and sensitive scenarios before release.
• Track production quality: Combine offline tests with real user feedback.

What Should You Evaluate?

Choose metrics based on the application. A customer-support assistant needs different tests from a document extraction system or research agent.

Offline vs Online Evaluation

Build a Golden Test Set

A golden dataset contains representative inputs and expected outcomes. Start small, then grow it from real failures and important business scenarios.

Python: Represent an Evaluation Case

Keep test cases structured so the same dataset can be reused after every change.

test_case = {
    "id": "billing-001",
    "question": "When is my invoice due?",
    "expected_facts": [
        "The invoice is due within 30 days."
    ],
    "must_be_grounded": True,
}

print(test_case["id"])

Store expected facts or acceptable answers instead of relying on one exact string match when multiple answers can be valid.

Exact Match and Rule-Based Checks

Deterministic checks are excellent when the expected output has a predictable structure.

Semantic Evaluation

For open-ended responses, two correct answers may use different words. Semantic evaluation checks whether the meaning is sufficiently aligned with the expected outcome.

LLM-as-a-Judge

An evaluator model can score qualities such as relevance, correctness, style, or groundedness. Give it explicit criteria and a structured output format.

Python: Simple Rubric

A rubric makes evaluation criteria explicit and easier to compare.

rubric = {
    "correctness": "0 = wrong, 1 = partly correct, 2 = correct",
    "relevance": "0 = off-topic, 1 = mostly relevant, 2 = direct",
    "groundedness": "0 = unsupported, 1 = partly supported, 2 = supported",
}

score = {
    "correctness": 2,
    "relevance": 2,
    "groundedness": 2,
}

print(sum(score.values()))

Evaluate RAG Retrieval Separately

Do not evaluate only the final answer. First check whether the retriever returned the information needed to answer the question.

If the required document never appears in the retrieved context, generation quality alone cannot fix the retrieval failure.

Retrieval Metrics

For search and RAG systems, evaluate retrieval independently using a labeled set of relevant documents.

• Recall@K: Did the relevant item appear in the top K results?
• Precision@K: How many of the top K results were relevant?
• MRR: How high was the first relevant result?
• NDCG: How well were results ordered when relevance has multiple levels?
• Empty retrieval rate: How often did search return nothing useful?
• Source coverage: Did the retrieved context contain the needed evidence?

Evaluate Agents

Agents need trajectory-level evaluation. A final answer can be correct even when the agent took an inefficient path, and an apparently reasonable path can still fail the task.

Evaluate Safety and Guardrails

Include adversarial cases in the evaluation set. Test whether the application blocks or safely handles unsafe inputs, prompt injection, sensitive-data requests, and unauthorized actions.

Human Evaluation

Human review remains valuable for ambiguous or high-impact tasks. Use a clear rubric so reviewers evaluate the same criteria.

Evaluation Thresholds

Turn scores into release decisions. Define minimum acceptable quality before the application reaches users.

For example, you might require retrieval Recall@10 to stay above a target while keeping cost and p95 latency below their budgets.

Regression Testing

Run the same golden dataset whenever you change prompts, models, embeddings, chunking, reranking, tools, or guardrails.

Compare Model Versions

Evaluate quality, latency, and cost together. A model with a slightly lower quality score may still be the better production choice for a low-risk task if it is dramatically faster and cheaper.

Production Evaluation

Offline evaluation does not represent every real-world request. Sample production traffic and evaluate it using privacy-safe telemetry, user feedback, automated checks, and periodic human review.

• Sample real requests by feature or tenant.
• Redact or minimize sensitive information.
• Track thumbs-up/down and task completion.
• Investigate low-score traces.
• Add important failures to the golden dataset.
• Watch quality trends after model or prompt changes.

Evaluation Pipeline

Keep evaluation automated enough to run in development and CI, while reserving human review for cases where automated scoring is not trustworthy.

Common Evaluation Mistakes

• Using only one test question.
• Measuring only model output and ignoring retrieval.
• Using exact string matching for open-ended answers.
• Trusting an LLM judge without validation.
• Ignoring safety and adversarial cases.
• Optimizing average quality while missing tail failures.
• Changing the test set until a model looks better.
• Collecting production data without privacy controls.

Mini Project: Build a RAG Evaluation Harness

Create a small Python evaluation harness for a support RAG application.

Evaluation Checklist