Enterprises are building agent systems with a familiar promise: “We’ll evaluate it before it reaches customers.” But a new wave of enterprise research suggests the real problem isn’t that teams aren’t running enough tests. It’s that the tests they trust don’t reliably predict what will happen in the messy, high-stakes reality of production.
In a June 2026 Pulse Research survey of 157 organizations with 100+ employees, the headline finding is stark: many enterprises are granting AI agents more autonomy than they trust their evaluations to justify. And the consequences aren’t theoretical. Half of respondents report that, within the past year, they shipped an agent or LLM feature that passed internal evaluations and then still caused a customer-facing failure. A quarter say it happened more than once.
That combination—“passed evals” and “customer failure”—is where the evaluation gap begins. It’s not simply a measurement gap. It’s a reality-alignment gap: the distance between what evaluation systems claim will happen and what actually happens when agents interact with real users, real data, real edge cases, and real business constraints.
What makes this especially urgent is the direction of travel. Two-thirds of organizations already allow, or are actively engineering toward, zero-human-in-the-loop deployment for low-risk agent changes based on automated evaluation results alone. In other words, the gate is being automated faster than the gate is being trusted.
This is the paradox at the center of the research: almost no one fully trusts automated evaluation today, yet most are moving toward using automated evaluation as the deciding factor for production autonomy.
A passing evaluation isn’t the same thing as a working agent
The survey asked a blunt question: in the past 12 months, did your organization deploy an agent or LLM feature that passed internal evaluations but then caused a customer-facing failure?
Half of organizations said yes. That means 50% have experienced the exact scenario that evaluation frameworks are supposed to prevent: the system cleared the internal bar, and then failed in front of customers anyway. Even more telling, 25% reported that it happened more than once.
Only 36% said they had no such failures. The remaining responses split into two categories: some organizations either don’t run pre-deployment evaluations (8%) or don’t track root causes closely enough to know (6%). Either way, the implication is the same: the evaluation verdict is not consistently mapping to real-world outcomes.
This is where the “coverage problem” framing starts to break down. If the issue were simply that teams need more test cases, more scenarios, more benchmarks, then you’d expect the solution to be straightforward: expand the suite, increase the number of evals, add more prompts, add more adversarial cases, and the failures should drop.
But the research points to something more structural. The most cited reason for low trust in automated evaluation is that evaluations align poorly with real-world outcomes. That’s not a “not enough tests” complaint. It’s a “the tests don’t represent the world” complaint.
When the gate doesn’t match reality, green checks become a false sense of safety
Trust in automated evaluation is extremely low. Only 5% of organizations say they fully trust automated evaluation as it stands. That means 95% identify at least one limitation that reduces trust.
The top limitation is the one that directly explains the “passed evals, failed customers” pattern: evaluations don’t align with real-world outcomes (29%). Bias or inconsistency follows (21%), and lack of explainability is also prominent (18%). Data leakage or privacy concerns appear as well (17%), which matters because evaluation pipelines often require collecting, transforming, and storing sensitive information—sometimes without the same rigor applied to production data handling.
If you zoom out, these limitations share a common theme: evaluation systems are not yet reliable enough to serve as the final authority for autonomy decisions.
Bias and inconsistency suggest that the same behavior can receive different verdicts depending on how the evaluation is run, what data is sampled, or how the scoring is interpreted. Lack of explainability suggests that even when an evaluation produces a pass/fail result, teams may not understand why. And if you can’t explain the verdict, you can’t confidently debug it, improve it, or decide whether it generalizes.
So the evaluation system becomes less like a safety mechanism and more like a probabilistic signal—one that teams may still use because it’s the only scalable option. But using a weak signal as a hard gate is exactly how you get the “green check, customer failure” loop.
The autonomy ceiling is rising anyway
Here’s the part that feels like it should be impossible, yet appears to be happening.
Even though almost no one fully trusts automated evaluation, two-thirds of organizations (66%) already allow, or are actively engineering toward, zero-human-in-the-loop deployment for low-risk agent changes based on automated evaluation results alone. Specifically, 34% already allow it for low-risk agents, and 33% are engineering pipelines to permit it within twelve months. Only 22% rule it out for the foreseeable future.
This is not just a small-company phenomenon. Larger enterprises are slightly further down the path toward zero human review than smaller ones (70% versus 64%), and they’re also slightly more likely to have shipped an evaluation-passing agent that then failed a customer (54% versus 48%). The sample size for the largest segment is smaller, so these comparisons should be treated directionally, but the overall message remains: the assumption that “more mature, regulated orgs keep humans in the loop longer” doesn’t hold up cleanly here.
The deeper issue is timing. Autonomy is arriving faster than assurance. And when assurance lags, the failure mode changes. Instead of occasional evaluation mismatches, you get systematic scaling of the mismatch—because the system is now allowed to change itself (or be changed) in production without a human checkpoint.
This is how an evaluation gap can widen over time. Not because teams stop caring, but because the pipeline design encourages speed and throughput, and the evaluation system is treated as sufficient even when it’s not trusted.
The evaluation stack is fragmented—and sometimes missing
If you’re wondering why enterprises can’t simply standardize on a single evaluation platform that everyone trusts, the survey offers a clue: the evaluation tooling landscape is fragmented and provider-led.
Provider-native tooling is common. OpenAI’s native evals and traces are used by 17% of organizations, and Anthropic’s Claude Console evals by 13%. But there’s also a striking answer: 17% of enterprises report using no dedicated agent-evaluation tooling at all.
That’s not a minor detail. If an organization is shipping agents to customers, having no dedicated evaluation tooling suggests either heavy reliance on ad hoc scripts, manual processes, or a belief that existing workflows are “good enough.” It also suggests that evaluation maturity varies widely across enterprises—even among those actively building agent systems.
Specialist evaluation vendors exist, but none has become a category standard. DeepEval is cited by 12%, Braintrust by 8%, and other tools appear in smaller shares. Some organizations build their own.
The result is a patchwork approach: different scoring methods, different datasets, different prompt templates, different rubric interpretations, and different ways of measuring “quality.” When the evaluation ecosystem is fragmented, it becomes harder to ensure that a pass verdict means the same thing across teams, models, and time.
And when the verdict isn’t consistent, trust naturally erodes.
Production monitoring often watches “functioning,” not “correctness”
Even if pre-deployment evaluation were perfect, production would still matter. Agents can drift. Data changes. User behavior shifts. Tool integrations evolve. Policies update. And models can behave differently under different contexts.
So the question becomes: when agents are live, do enterprises monitor whether outputs are correct—or only whether the system is functioning?
The survey draws a sharp distinction between two types of monitoring:
Functioning monitoring asks: is the agent up and responding? Did requests complete? How fast? At what cost? Were there errors?
Correctness monitoring asks: are the outputs right? Did the agent take the right action? Did it stay within policy?
The split is stark. 51% of organizations monitor only whether the agent is functioning. 23% monitor whether answers are right. When you include ad hoc reviewers and don’t-knows, roughly three-quarters of organizations are not running automated, real-time evaluation of output correctness in production.
This is a runtime mirror of the pre-deployment gap. If you don’t measure correctness continuously in production, then many failures will look like success until someone notices. A response can be fluent, fast, and error-free while still being wrong, unsafe, or misaligned with user intent.
That means enterprises may be blind to the very signals that would help them calibrate their evaluation systems. Without real-time correctness monitoring, it’s harder to learn from failures, quantify how often evaluation passes correlate with production correctness, and adjust the evaluation pipeline accordingly.
In effect, many organizations are relying on uptime and cost metrics as proxies for quality. Those proxies can keep systems stable, but they don’t guarantee truthfulness, policy compliance, or task success.
How enterprises choose eval tools: cost and integration first, consistency as the goal
Another interesting angle in the research is how enterprises select evaluation tooling and what they treat as success.
Enterprises buy evaluation tooling primarily based on economics and integration. Cost of evaluations is the top selection driver (28%), narrowly ahead of ease of integration (27%) and evaluation accuracy (24%). Breadth of observability (13%) and vendor roadmap (4%) matter less.
On what success looks like, evaluation consistency dominates. More than a third (36%) name evaluation consistency—getting the same verdict on the same behavior every time—as the primary measure of success. That’s ahead of speed of experimentation (19%), reduction in failures (18%), production visibility (13%), and compliance (11%).
This is revealing because it connects directly back to the trust limitations. Bias or inconsistency was one of