Is your AI agent actually working?

The short answer: measure task completion, not response time. Run cheap deterministic checks on everything, expensive judgement-based scoring on a small sample, and keep a fixed set of real past cases you re-run whenever anything changes. Gartner projects that by 2028, 40% of enterprise AI failures will trace to inadequate evaluation and monitoring rather than to model capability — which matches what we see: the model is rarely the problem, and nobody was watching.

Third-party figures on this page were checked on 26 September 2026 against the publishing organisation's own release. Research in this field ages quickly and definitions vary between surveys — treat any undated statistic with suspicion, including on our own pages.

Why silent failure is the normal failure

Conventional software fails loudly. It throws, it 500s, it stops. An agent fails by succeeding at the wrong thing: it summarises the wrong document, it answers from a policy that changed in March, it books the meeting in the wrong time zone, it reassures a customer about something it has no information on.

Every one of those produces a fast, well-formed, confident response. If your monitoring measures whether a response was produced, all of them pass.

The three layers, cheapest first

Layer 1: deterministic checks, on everything

Rules, not judgement. They cost nothing and catch a surprising amount:

• Did the agent call the tool it needed to, or answer from memory when it should have looked something up?
• Did it produce the required structure — a reference number, a date, a link — when the job demands one?
• Did it finish, or stop mid-task?
• Did it stay inside its boundary, or attempt something that needed approval?
• Is the output length wildly outside the normal range? Both directions are signals.

Run these on 100% of traffic. They are the smoke alarm.

Layer 2: judgement-based scoring, on a sample

For the things a rule cannot check — was the answer correct, was it responsive to what was actually asked, was the tone right for the channel. The standard approach is to have a model score the output against written criteria, which works well enough to be useful and badly enough that you should not run it unsupervised.

Two disciplines make it worth doing. Sample rather than scoring everything — common practice is 5–20% of production interactions, which bounds both cost and variance. And have a human review the scorer periodically, because a judge that has drifted is worse than no judge: it produces a dashboard that is confidently wrong.

Layer 3: a regression set of real cases

This is the layer people skip and the one that pays back most.

Keep twenty to fifty real past interactions — the tricky ones, the ones that went wrong, the ones a customer complained about — with the answer you consider correct. Re-run the whole set whenever anything changes: a new model, an edited instruction, a new tool, a knowledge-base update.

Its value is that it catches the failure mode nobody expects: fixing one thing quietly breaking another. Without it, that is discovered by a customer.

What to actually measure

Four numbers cover most business deployments:

1. Task completion rate. Of the things it was asked to do, how many were done correctly, end to end. The headline number, and the one most teams do not have.
2. Escalation rate, and whether it is right. Both directions are failures: escalating what it could handle wastes the point of the agent; handling what it should have escalated is the expensive one.
3. Human edit rate on drafts. If every draft is heavily rewritten, the agent is generating work rather than removing it. A slow rise here is the clearest early warning of drift.
4. Time-to-resolution against the previous baseline. The business number. Without it you cannot answer whether any of this helped.

Latency and token usage still matter — for cost and for user experience — but they are operational metrics, not quality ones. Treat them as such. The cost side is in what an agent costs to run.

The weekly review, which is the whole practice

None of the above works without someone looking. Fifteen minutes per agent per week:

• Scan what it actually did, not a summary of what it did.
• Read three outputs in full, chosen at random rather than by a filter.
• Check the four numbers against last week.
• When something is wrong, fix the cause rather than the instance — a repeated wrong answer usually means a wrong fact in the agent's context, not a stubborn model.

That last habit is what separates a deployment that improves from one that decays. It is also the cheapest quality process available, and the reason it is skipped is that nobody owns it — which is failure mode two in why agentic AI projects get cancelled.

When to widen the boundary

Evaluation is what makes that decision evidence-based rather than nervous. A category of action is ready to run unattended when its completion rate has been high for a sustained period, its edit rate is low, and you can see in the log what it has been doing. Until then it stays in the approval queue — not as a permanent state, but as the thing you are collecting evidence to change.

And some categories never move, regardless of the numbers, because the cost of being wrong once exceeds the accumulated saving. That is a judgement call, not a metric, and it belongs to a person. Approvals and trust.