Step 1: Write down the number the agent was bought to move
An agent with a 95% eval score and no changed process is performing at zero. That is the uncomfortable part of this question. Most agent dashboards report tokens, latency, and an offline eval percentage, and none of those three tell you whether the work got done. So start with a sentence, before any instrumentation: "This agent succeeds if [metric] moves from [baseline] to [target] by [date]." A support agent bought to cut first-response time is measured in minutes of first-response time. A claims agent is measured in claims closed per adjuster per day. If nobody on the team can fill in those four blanks, you don't have a use case yet. You have a demo with a budget. We've watched this go wrong in the same way repeatedly: the metric gets chosen after launch, once someone asks for a number, and by then it is chosen to flatter whatever the agent already does. Outcome of this step: one named metric, one owner, one target, one date, agreed in writing before the agent touches real traffic.
Step 2: Capture the baseline before the agent starts
You cannot measure a delta you never took the first half of. Give yourself two to four weeks of pre-agent data on the metric from step 1: the current handling time, the current error rate, the current cost per case. Pull it from the systems that already run the process, not from a manager's estimate. This is the step teams skip most, and it's the one that makes every later number arguable. Once the agent is live, the baseline becomes a reconstruction, and a reconstructed baseline is a negotiation. Keep the segment honest too. If the agent only handles the routine 30% of tickets, the baseline is the routine 30% of tickets, not the whole queue. Comparing a cherry-picked agent segment against a full-queue average is the most common way an agent looks like a win on a slide and like nothing in the P&L. Outcome: a dated baseline file, scoped to the exact segment the agent will handle, that a finance partner would accept.
Step 3: Score task success on real traffic, not accuracy on a test set
Offline evals are hiring tests. They tell you the agent is capable of the job. They tell you nothing about how it performs on Tuesday afternoon with a customer who typed three questions into one message. Build a task-level rubric instead, and score real runs against it. For each run, one binary: did this fully resolve the task without a human redoing it? Then a small set of reasons for the failures (missing context, wrong tool call, refused when it should have acted, acted when it should have refused). Sample 50 to 100 real runs a week and have a human who knows the process score them. Yes, by hand. The rubric is worth more than another benchmark. The number that matters here is unassisted task completion rate on production traffic. Track it weekly, per segment. Expect it to sit well below your eval score, and expect the gap itself to be the most useful thing you learn. Outcome: a weekly unassisted completion rate with a labelled failure breakdown you can act on.
Step 4: Make every score traceable back to what happened
A score you can't explain is a score you can't defend, and in regulated work it's a score you can't ship. When an agent action is scored a failure, you need the whole chain: the input, the context it retrieved, the tools it called, what it returned, and who or what accepted the result. This is also where the standards land, and they land as supporting evidence rather than the point. ISO/IEC 42001 Clause 9.1 asks an organization to decide what it monitors and measures, by what method, and when. The EU AI Act's Article 72 post-market monitoring duty asks providers of high-risk systems to collect and review performance data across the system's life, and Article 15 puts accuracy and its declared metrics on the record. Both are satisfied by the same trace you need anyway to debug a bad answer. Outcome: any scored run can be reconstructed end to end, months later, by someone who wasn't there.
Step 5: Report value net of what it costs to run
Gross value is not performance. An agent that saves 400 hours a quarter and costs more than 400 hours of the same labour to run and supervise is a performance problem wearing a success costume. So track two lines next to each other, per segment, per month: the value the metric from step 1 actually moved, and the fully loaded cost of the agent. Fully loaded means inference spend, retrieval and storage, the humans reviewing outputs, and the engineering time keeping it alive. Human review is the line that surprises people. It is often the largest one, and it usually grows quietly as the agent takes on more traffic. Outcome: a net value per segment that survives contact with a CFO.
Step 6: Read the failures every week and let them move the roadmap
Put a standing 30 minute review on the calendar with the process owner, the engineer, and whoever scores the rubric. Read the failed runs. Not a summary of them, the actual runs. Three things come out of a review like that, reliably. Some failures are context problems, and they get fixed with better retrieval or a tool the agent didn't have. Some are scope problems, and the honest fix is to narrow what the agent is allowed to attempt. Some are use-case problems, and the honest fix is to stop. Operators who have taken AI to production at 300,000-organization scale will tell you the third category is the one that pays, because a use case retired in month two is budget released rather than burned. The agents that reach production and stay there in financial services, healthcare, and government are almost never the ones with the best model. They're the ones whose owners could show, on demand, what the thing did and what it was worth. Outcome: a dated decision log of what changed after each review, and at least one thing narrowed or stopped.
Frequently asked questions
What is the single best metric for AI agent performance?
There isn't one metric that travels across agents, but there is one shape: the business number the agent was bought to move, measured against a baseline captured before launch, in the same segment the agent handles. Unassisted task completion rate on production traffic is the best general-purpose companion to it.
Are offline evals worth running at all?
Yes, as a gate before an agent touches real traffic, and as a regression check when you change a prompt, a model, or a tool. Treat the eval score as a capability check. It is not evidence of performance, and reporting it to a board as though it were is how AI programmes lose credibility.
How long before an agent's performance can be judged?
Plan on two to four weeks of baseline before launch, then four to six weeks of production traffic before the numbers mean much. Anything read earlier is mostly novelty effect, in both directions: users try to break it, and users also give it easier work than they give a colleague.
Who should own agent performance measurement?
The process owner, not the AI team. If the person accountable for claims handling time doesn't own the agent's scorecard, the scorecard drifts toward metrics the AI team can win. That is the failure mode to design against.