← the delivery methodology

agent evaluation framework

Six gates between a working demo and an agent you trust.

Each gate answers one question, produces evidence a risk committee can read, and has a bar agreed before the numbers come in. The gates keep running after go-live, because models, prompts and inputs keep changing.

the six gates

In order. A gate that fails sends you back, usually to the golden set or the step it points at, not forward with a caveat. The bars are starting points; agree yours with the people who own the risk before you measure.

  1. 01

    define done

    What does a correct output look like, and who says so?

    Write down, per step and end to end, what correct means: which fields must match exactly, which can be paraphrased, which errors are critical and which are cosmetic. Then build the golden set: real cases, labelled by the people who do the work today, with the hard ones in it on purpose.

    Two people label a sample independently first. Their agreement with each other is the ceiling: no model can be expected to agree with one reviewer more than a second reviewer does.

    measure
    Labeller agreement (Cohen's kappa) on a double-labelled sample; golden set size and mix.
    passes when
    Kappa between labellers of 0.8 or more on critical fields. At least 300 cases, with every known exception type represented, and between a fifth and a third of them hard.
    evidence
    The rubric, the golden set with its version, and the double-labelling results.
    work it out with
    AI accuracy proof calculator: how many cases prove the accuracy you need
  2. 02

    offline evals

    Is each step good enough on its own, and is the chain?

    Run every step against the golden set and score it: field-level accuracy for extraction, precision and recall for classification, the share of critical errors for judgement steps. Then run the whole pipeline end to end, because errors compound: five steps at 95% are right about 77% of the time together.

    Report the lower end of the confidence interval, not the point estimate. A step that is 97% right on 100 cases may be 92% right in truth.

    measure
    Per-step accuracy with Wilson intervals; end-to-end straight-through, escalation and silent-error rates; cost and latency per case.
    passes when
    Lower bound of each critical step at or above its target. Silent-error rate end to end below the rate the current manual process achieves.
    evidence
    An eval report per step and for the chain, tied to model, prompt and golden-set versions.
    work it out with
    Agent reliability calculator: how step accuracy, checks and retries add up end to end
  3. 03

    calibrate the judge

    Can an automated grader be trusted to score what people can't read?

    Golden sets cover hundreds of cases; production runs thousands. An LLM judge scores the rest, but only after it has been measured against people on the same cases. Score agreement, and above all the share of real failures the judge catches.

    Agreement looks high when most cases pass. Kappa corrects for that; the catch rate tells you directly how many failures slip past.

    measure
    Cohen's kappa between judge and human labels, with an interval; judge catch rate on known failures; false alarm rate.
    passes when
    Kappa of 0.6 or more for a judge that informs people; 0.8 or more, with a catch rate of 90% or more, before a judge gates anything automatically.
    evidence
    The judge prompt and version, the agreement report, and the disagreements reviewed by a person.
    work it out with
    LLM judge agreement calculator: kappa, catch rate and how many cases to label
  4. 04

    shadow run

    Does it hold up on live traffic, next to the people who do it today?

    Run the pipeline on real incoming cases for two to four weeks without acting on its output. People keep doing the work; the system's answers are compared with theirs. This is where data drift, odd formats and the cases nobody put in the golden set show up.

    Use the shadow data to set confidence bands: measure how often the model is right at each confidence level, and choose the threshold below which cases go to a person.

    measure
    Agreement with human outcomes by case type; accuracy per confidence band; escalation rate; cost and cycle time per case.
    passes when
    No case type worse than the manual baseline on critical errors. Accuracy falls as confidence falls (the model is calibrated). Projected review load fits the team.
    evidence
    The shadow comparison report, the confidence-band table, and the threshold chosen with its cost.
    work it out with
    Escalation threshold calculator: where to set the hand-off to a person
  5. 05

    staged rollout

    What happens when it starts acting on real cases?

    Let the system act on a small slice first (one case type, one team, or 5% of volume) with every output reviewed. Widen the slice only when the bar for the current one has held for an agreed period. Each stage has written stop conditions, and anyone on the team can pull it.

    Staff the review queue for the escalation rate you measured, plus a margin, or the queue becomes the new bottleneck.

    measure
    Reviewer override rate; critical errors found in review; escalation rate against forecast; queue wait time.
    passes when
    Override rate below the agreed bar (often 2% to 5%) for two consecutive weeks before each widening. Zero unresolved critical errors.
    evidence
    The rollout log: each stage's dates, volumes, metrics and the sign-off to widen.
    work it out with
    Human review staffing calculator: reviewers needed for the escalation volume
  6. 06

    monitor and re-prove

    Is it still as good as the day it went live?

    Sample live outputs for human review every week, score the rest with the calibrated judge, and watch the input mix for drift. Every change to a model, a prompt, a tool or a policy re-runs the golden set before release, in CI, and fails the build if a critical metric drops.

    Add every production error that review finds to the golden set. Over a year the set becomes the best description of the work there is.

    measure
    Weekly sampled accuracy; judge-scored accuracy; input drift; escalation and override rates; cost per case.
    passes when
    Alert when sampled accuracy's lower bound drops below the go-live bar, or escalation moves more than a third from its baseline. Golden-set regression blocks a release.
    evidence
    The monitoring dashboard, the weekly sample results, and the regression report attached to each release.
    work it out with
    Agent loop cost calculator: catch cost drift as tasks take more turns

what goes in the golden set

Every gate leans on it, so it decides how much the others are worth.

  • Real cases from the last few months, not synthetic ones, with personal data handled under the same rules as production.
  • Every exception type the team knows about, even the rare ones, at a higher share than in live traffic.
  • Cases the current process got wrong, with the corrected answer: these show whether the system repeats people's mistakes.
  • Labels from the people who do the work, against a written rubric, with the disagreements resolved and recorded.
  • A frozen version for go-live decisions and a growing version for monitoring. Never tune prompts on the cases you report on.

what to re-run when something changes

Passing once is not the end. These are the changes that most often break an agent that used to work, and the gates each one sends you back through.

changere-run gates
New model or model version02, 03, 04 (short), 06 baseline reset
Prompt or tool definition change02, and 03 if the judge reads that step
New document type or case type01 (extend the golden set), 02, 04 for that type
Policy or threshold change02 end to end, 04's band table re-read
Drift alert or error spike06 investigation, then 01 and 02 with the new cases

where it fits

bring an agent that needs proving

A pilot that looks good, a vendor claiming 95%, or a risk committee asking for evidence. In 15 minutes we can work out which gates you have already passed and which you haven't.