How to evaluate an AI workflow before deployment

A polished demo answers the examples chosen for it. An evaluation asks whether the workflow survives representative, ambiguous and hostile cases while still improving the business.

That requires more than scoring model responses. Evaluate the whole chain: input, retrieval, model, tools, human review, final action and recovery.

Begin with the decision the system supports

Write a one-sentence job:

From an inbound service request, extract the requested service and location, propose a routing category, show the supporting text and send low-confidence cases to dispatch.

Then state what the system must never do, such as invent an address, expose another customer's information or confirm a booking without capacity.

Vague goals such as "improve efficiency with AI" cannot be evaluated.

Establish the current baseline

Measure the existing workflow on quality, time, cost and failure. If people currently route 95% of requests correctly in four minutes, an AI workflow that achieves 90% in two minutes is not an obvious win. Consequences and correction time matter.

Record volume, exception rate, wait time, staff effort, rework and meaningful outcome. Compare the new workflow against this baseline, not against doing nothing in a laboratory.

Build a representative test set

Use real, appropriately handled examples where permitted, then add designed cases. The set should cover:

• common cases;
• rare but important cases;
• missing and contradictory information;
• different languages, formats and writing styles the business sees;
• outdated or conflicting source documents;
• attempts to manipulate instructions;
• tool and integration failures;
• cases where the correct response is to abstain or escalate.

Keep a separate holdout set that was not used while tuning prompts or rules. Version the data, expected result and evaluator guidance.

Score dimensions separately

A single average hides dangerous tradeoffs. Measure:

1. Task correctness: did it reach the right result?
2. Groundedness: are claims supported by approved evidence?
3. Completeness: are material conditions present?
4. Abstention and escalation: does it stop in the right cases?
5. Tool correctness: were the right actions and parameters selected?
6. Privacy and security: did it avoid unauthorized data and instructions?
7. Human usability: can a reviewer understand and correct it?
8. Latency and reliability: does it complete within operational limits?
9. Business outcome: does it reduce time, error or cost without shifting harm elsewhere?

Set thresholds by consequence. A small wording defect and a wrong account update should not carry the same weight.

Combine automated and human evaluation

Exact-match checks work for IDs, calculations and required fields. Code can verify schemas, permissions and prohibited actions. Model-based grading can help scale review, but it is itself a model output and should be calibrated against people.

Subject-matter experts should evaluate samples with a written rubric and examples. Track agreement between reviewers. Google Cloud's model-evaluation documentation similarly supports pointwise, pairwise and computation-based metrics rather than relying on one measure. Google Cloud

Test the system boundaries

For an agent, simulate unavailable tools, timeouts, repeated events, revoked permissions, stale records and malicious content. Confirm spending, action and time limits. Verify that retries are safe and actions are logged.

The Government of Canada's agentic AI guide recommends sandbox testing, bounded permissions, audit trails, red-teaming, recovery and staged deployment. Government of Canada

Pilot in increasing stages

A sensible sequence is:

1. offline evaluation with no real actions;
2. shadow mode beside the existing process;
3. internal users on low-consequence tasks;
4. approval-required actions with close review;
5. limited automation within explicit thresholds;
6. broader deployment only after exit criteria pass.

Define rollback triggers before launch. Examples include a security event, a rise in serious error, unexplained drift or reviewer workload above capacity.

Monitor after deployment

Deployment changes the input distribution. Policies, user behaviour, vendors and models change too. Monitor quality by segment, exception volume, corrections, overrides, complaints, latency, cost and security events. Re-run the evaluation set when prompts, data, tools or models change.

NIST's AI Risk Management Framework calls for pre-deployment testing, regular monitoring, documented test sets and safe failure beyond system limits. It organizes the work through Govern, Map, Measure and Manage functions. NIST AI RMF Core

Keep an evaluation record

Record versioned components, dataset provenance, known gaps, thresholds, results, reviewers, approvals and monitoring plan. State where evidence is weak.

The purpose is not to prove that an AI system is "safe." It is to collect enough evidence to justify a bounded use, detect when conditions change and stop when the evidence no longer holds.

Source check

NIST and Government of Canada materials are risk-management guidance, not certification. Evaluation thresholds and test cases must reflect the actual workflow, affected people, law and consequences.

Sources

• NIST AI RMF Core
• NIST: Generative AI Profile
• Government of Canada: Guide on the use of agentic artificial intelligence
• Google Cloud: View and interpret evaluation results