LLM Evaluation: How to Measure Production Accuracy

To evaluate an LLM system, start with representative tasks and a written definition of a correct result. Measure retrieval separately from answer quality, check important failure types, and compare each change with the same baseline. A single average accuracy score can hide the errors that matter most to your team.

For a customer-support assistant, an unsupported refund promise may matter more than an awkward sentence. For document extraction, a wrong total may matter more than a missing optional field. Agree those differences before choosing a metric.

Evaluate retrieval and answers separately

For a RAG system, first check whether the search step returns the information needed to answer the question. Use stable document or chunk identifiers and reviewed relevance labels.

• Recall: how much of the known relevant material was retrieved?
• Precision: how much of the retrieved material was relevant?

The small example below measures sets of relevant chunk IDs. It does not measure ranking quality. Questions with no relevant source need a separate test of whether the system declines to answer.

def evaluate_retrieval(expected_ids: list[str], retrieved_ids: list[str]) -> dict:
    expected = set(expected_ids)
    retrieved = set(retrieved_ids)
    if not expected:
        raise ValueError("Evaluate no-source questions separately")
    matches = expected & retrieved
    return {
        "recall": len(matches) / len(expected),
        "precision": len(matches) / len(retrieved) if retrieved else 0.0,
    }

Then review the answer itself:

• Faithfulness: are its claims supported by the retrieved material?
• Relevance: does it address the question?
• Completeness: does it cover the required parts?
• Abstention: does it avoid inventing an answer when evidence is missing?

Ragas faithfulness evaluation is one option for automated judging. An LLM judge is another model, not ground truth. Compare its judgments with human-reviewed examples, record its model and prompt, and review disagreements. Pin library versions and follow the documentation for that version rather than mixing examples from different releases.

Choose metrics for the actual task

For classification, report precision and recall for each important class. Overall accuracy can conceal poor performance on a rare but costly class. For extraction, measure each field and distinguish missing values from incorrect ones. For summaries, review factual support and required coverage; word overlap alone does not establish correctness.

Record latency, cost and human-review rate alongside quality. A result that needs frequent correction may not save time even if its average score looks good.

Build two kinds of test set

Keep a versioned, held-out set for comparing releases. Do not tune prompts against those answers repeatedly and still treat them as unseen data. Use a separate development set for experiments.

Also maintain a regression set. Add reviewed production failures to it so that a fix stays fixed. Growing this set is useful; it is different from quietly changing the benchmark used to compare two releases.

Include empty inputs, conflicting documents, ambiguous questions, permission boundaries and requests outside the supported task. Break results down by use case, language and other relevant groups.

There is no universal minimum of 50 or 100 examples. The required sample depends on the error rate, the difference you want to detect and the risk of missing a failure. Report the sample count, sampling method and uncertainty. A small pilot can reveal obvious problems without proving that rare failures are acceptably uncommon.

Make release checks explicit

Agree acceptance criteria before looking at the final results. For example, you might require no known permission leaks in the security test set and a task-specific quality threshold on reviewed answers. Passing a finite test set does not prove that a failure is impossible.

Store these details with each evaluation:

• Dataset version and number of examples.
• Model, prompt, retrieval settings and source-document version.
• Scoring rules and judge configuration, if used.
• Results by task group, plus failed examples and reviewer decisions.

Use those results in the release check. Averages should not override a critical failure, and empty or failed evaluation runs should block release rather than count as a pass.

Monitor after deployment

Production questions and source documents change. Sample permitted production examples on a schedule that matches usage and risk, then review quality against a comparable baseline. Missing labels are a limitation to report, not a reason to invent reference answers.

Avoid putting full questions, retrieved documents or answers into routine logs by default. Agree what may be retained, redact sensitive content and restrict access. Hashing a query does not automatically make it anonymous or preserve enough information to evaluate the answer.

This illustrative helper flags a drop greater than five percentage points for scores on a 0–1 scale. Five points is an example alert threshold, not a recommended accuracy target. The inputs must use the same scoring definitions and comparable samples.

from math import isfinite

def check_drift(current: dict, baseline: dict, threshold: float = 0.05) -> list[str]:
    if not 0 <= threshold <= 1:
        raise ValueError("Threshold must be between 0 and 1")
    if not baseline or current.keys() != baseline.keys():
        raise ValueError("Expected the same non-empty set of metrics")
    regressions = []
    for metric, previous in baseline.items():
        score = current[metric]
        if not all(isfinite(value) and 0 <= value <= 1 for value in (previous, score)):
            raise ValueError("Scores must be finite values between 0 and 1")
        if previous - score > threshold + 1e-12:
            regressions.append(metric)
    return regressions

An alert should lead to investigation, not an automatic rollback. Check changed documents, broken ingestion, new question types and model or prompt changes. Inspect the failed examples before deciding what to repair.

What to agree before an evaluation engagement

Bring a sample of the tasks, the source material the system may use and examples of unacceptable answers. Agree the review process, acceptance criteria and who handles exceptions. Quality targets from another client are not a benchmark for your system.

Our Enterprise RAG Knowledge System case study shows a related implementation. For an internal knowledge workflow, see RAG pipeline development and Operations Automation.

If you need help defining or testing an LLM workflow, discuss your project. The first conversation is about requirements and fit, not a promise of a free evaluation.

Related: RAG Pipelines in Production | Fine-Tuning vs RAG | AI & Machine Learning services