Two agents, identical scores, one of them is a liability
Put two agents side by side. Both score 91 percent on the same held out evaluation set. One was built against a data source refreshed nightly with a schema contract and an owner; the other reads a spreadsheet a team exports by hand when someone remembers. Both are equally ready by the benchmark and nowhere near equally ready for production. The evaluation measured the model's behaviour on a frozen sample of the world. Production is the unfrozen version. That gap is the reason so many pilots pass their review and then fail in month three, and it's why AI production readiness is better treated as a question about evidence than a question about score. The useful reframing: readiness is not how well the system performed once, it's how confidently you can say it will keep performing, and how fast you would know if it stopped.
What the standards actually ask you to produce
Read the three reference documents most enterprises are working against and a pattern shows up. None of them ask for a benchmark number. The NIST AI Risk Management Framework 1.0, published January 2023, organises itself around four functions: GOVERN, MAP, MEASURE and MANAGE. MEASURE is one of four, and it sits between mapping the context and managing the system in operation, which is a deliberate ordering. ISO/IEC 42001:2023 is a management system standard, so what it certifies is that you run a repeatable process with defined roles, documented risk treatment and internal audit, not that any one model hit a threshold. The EU AI Act goes furthest in naming artefacts: Article 12 requires automatic logging over the lifetime of a high risk system, Article 14 requires human oversight that a person can actually exercise, Article 17 requires a quality management system, and Article 72 requires post market monitoring after the thing is live. Every one of those is about the period after deployment. The benchmark culture has readiness backwards, testing hardest at the moment of least consequence.
Five classes of evidence that hold up
In practice the artefacts that separate a ready system from a scored one fall into five groups. Data lineage: you can name the source of every input, who owns it, how often it changes, and what happens when it does not arrive. Behavioural bounds: you have written down what the system is not permitted to do and can demonstrate the boundary holds, rather than trusting a prompt to hold it. Oversight records: a named human can intervene, and there is a record of when they did and what they changed, which is what Article 14 means by effective oversight rather than a checkbox. Drift detection: something watches inputs and outputs against the distribution you validated on, and alerts a person, not a dashboard nobody opens. Rollback: you can move back to the previous model or configuration inside a defined window, and you have done it at least once as a drill. A team that can produce all five for a workflow has a stronger readiness case than a team with a higher eval score and none of them.
The regulatory clock moved, and the readiness case did not
One correction worth making, because a lot of published advice is now stale. The obligations for Annex III high risk systems under the EU AI Act were widely written up as arriving in August 2026. The Digital Omnibus package pushed that back, with Annex III high risk obligations deferred toward December 2027. That extra runway has been read by some teams as a reason to slow down, which is the wrong lesson twice over. Penalties for prohibited practices still reach 35 million euro or 7 percent of global annual turnover, and general purpose model obligations have already been in force since August 2025. More to the point, none of the five evidence classes above exist because a regulator asked for them. They exist because systems without them break quietly. The deadline moving changes when an auditor knocks. It does not change when your data source silently changes format.
The uncomfortable recommendation
Here is the part practitioners argue with: stop gating production on a model quality bar and start gating it on an evidence bar. A system that scores 84 percent with full lineage, bounded behaviour, recorded oversight, drift alerting and a rehearsed rollback should ship ahead of one that scores 93 percent with none of it. The objection is that this lets weaker models into production. That is exactly the trade, and it's the right one, because the 84 percent system's failure modes are visible and recoverable while the 93 percent system's are neither. Model quality improves on its own as vendors ship new versions. Evidence does not appear on its own, and retrofitting it into a live workflow costs several times what building it in cost. Operators who have taken AI to production at 300,000 organisation scale tend to arrive at the same conclusion from the other direction: the incidents that hurt were never the ones where the model was slightly worse than expected.
How to run the assessment
Take one workflow you intend to put live in the next quarter and score it out of five on the evidence classes, with a named owner and an artefact for each. Anything you cannot point at a document or a system for scores zero, regardless of how confident the team is. Most first passes come back at two, and the two that are usually present are lineage and some form of logging. Missing almost everywhere: a rehearsed rollback and an oversight record that would satisfy someone who was not in the room. Fix the rollback drill first, because it's the cheapest and it changes the risk posture of everything else. Then work the oversight record, since it doubles as the acceptance data you need to prove the workflow's value. Readiness built this way gives you the same evidence base for a board conversation, a customer security review and an eventual conformity assessment, which is three uses for one piece of work.
Primary sources
NIST AI Risk Management Framework 1.0 (NIST AI 100-1, January 2023), particularly the MEASURE and MANAGE functions and the companion Playbook. ISO/IEC 42001:2023, Information technology, Artificial intelligence, Management system, with Clause 6 on planning and risk treatment and the Annex A controls. Regulation (EU) 2024/1689, the EU AI Act, Articles 12, 14, 17, 26 and 72 for logging, human oversight, quality management, deployer obligations and post market monitoring, read alongside the Digital Omnibus amendments to the Annex III timeline. Read the Act's articles directly rather than a vendor summary of them, because the summaries almost all lead with penalties and skip the operational obligations, which are the part that changes what you build.