How to Evaluate an AI System Before and After Launch

AI quality cannot be reduced to whether a response sounds convincing. A production system succeeds when it helps users complete a real task consistently, safely, and within acceptable cost and latency.

Begin with the decision the system supports

Define the user, input, expected result, and action that follows. Evaluation criteria should reflect that workflow. A support assistant may need grounded answers and correct escalation. A document system may need field-level accuracy and traceable source evidence.

Build a representative evaluation set

Collect examples from the actual operating environment. Include frequent requests, unusual formats, incomplete information, ambiguous cases, adversarial inputs, and situations where the system should refuse or hand control to a person.

Keep a protected test set that is not used while adjusting prompts or retrieval. Otherwise, improvements may only reflect memorization of familiar examples.

Score the dimensions separately

A single quality score hides useful information. Measure the dimensions that can fail independently:

• factual correctness and source support
• task completion and instruction following
• correct tool selection and arguments
• policy and permission compliance
• appropriate refusal or escalation
• latency and cost per completed task

Different failures require different fixes. Poor evidence may point to retrieval. Incorrect actions may indicate weak tool boundaries. Slow responses may require a smaller model or a redesigned workflow.

Combine automated checks with human judgment

Deterministic checks work well for schemas, citations, required fields, tool calls, and known answers. Model-based graders can help compare outputs at scale, but they should be calibrated against human review.

People remain important for nuance, usefulness, tone, and high-impact decisions. Use a clear rubric so reviewers judge the same qualities consistently.

Test the system, not only the model

Production behavior includes retrieval, prompts, tools, permissions, retries, user interface, and escalation. Evaluate complete scenarios from input through final action. A strong model can still fail inside a weak system.

Continue evaluation after launch

Real usage will reveal cases the test set missed. Monitor corrections, abandoned tasks, escalation rates, latency, cost, tool failures, and user feedback. Add important failures to the evaluation set before changing the system.

The goal is not a permanently perfect score. It is a dependable process for detecting regressions, understanding tradeoffs, and improving the outcomes that matter.

Explore how evaluation fits into production AI product development and AI infrastructure and MLOps.