1. Who we talk to
We spend our days in conversation with the people doing this work: heads of AI, directors of engineering, CTOs, and platform leads at consumer marketplaces, global banks, fintechs, security vendors, and data companies. Five problems come up so consistently, across companies that share nothing else, that they read less like separate complaints and more like the industry describing a single condition.
2. Agents ship without evaluation
The most consistent finding: agents are already live in production at almost every company we spoke with, and evaluation still trails behind that reality. A data scientist who consults across several large enterprises put the typical state bluntly: most of the companies he works with had no systematic evaluation on day one, and the ones that did had an ad hoc human labeling process at best. What strikes us is that nobody hides this. A product leader at a consumer software company described his own team's approach, without embarrassment, as shipping whatever worked fastest and calling it good enough. Evaluation debt is common enough that admitting to it carries no stigma.
Where evaluation exists, it rarely maps to what the business cares about, and the failure runs in both directions. At one ride-hailing and delivery marketplace, leadership rejects proxy metrics like pull-request throughput standing in for revenue, but engineering has nothing better to offer in their place. At a search and recommendations company, the blockage sits upstream of measurement entirely: getting business stakeholders to agree on what "good" even means turned out to be harder than any of the technical work underneath it. Engineering lacks a number finance will accept, and the business side often cannot name the number it wants moved.
The cost of that gap gets quantified rarely, but one shopping-assistant team did the math. The majority of its pre-launch eval spend had gone to synthetic tests with almost no predictive power, while a fifth of real user traffic turned out to be questions nobody had designed for, invisible until a staged rollout exposed them. The metrics the team ended up trusting were the unglamorous ones: carts created, orders completed, and an order-value lift of about twelve percent against each user's baseline. That last number carried weight in the boardroom for one reason: a competitor was publicly claiming thirty-five, and leadership wanted to know which world they lived in.
3. Cost visibility before cost control
Token and compute costs come up in nearly every conversation, and the shape of the problem repeats. A company realizes it is spending seven figures a month on model calls, with no attribution by team or project, before anyone builds a system to control it. One infrastructure lead, at a mid-sized SaaS company, described routing everything through a single API gateway only after the spend had already become a board-level concern. Another team, at a developer-tools company, watched a single automated test agent burn tens of thousands of dollars in an afternoon, a number that became the story every new hire gets told.
Evaluation itself is often the single biggest line item in the budget, not a rounding error. At more than one company, over half of the entire agent development budget went to evaluation before a single release, driven by manual review requirements that scale with every new agent shipped. The companies furthest along have converged independently on the same fix: a small team building a routing layer that sends easy tasks to cheap models and hard tasks to expensive ones, with response caching so the same question is never paid for twice inside a short window.
The counter-moves are getting specific, and cheaper than most teams assume. One team, at a workflow-automation software company, fine-tuned an eight-billion-parameter model to frontier parity on a narrow, labelable task for roughly a thousand dollars of training compute, cutting that workload's model bill by more than half. The thousand dollars is the interesting part: this technique is no longer reserved for companies with research labs, and what blocks it is rarely the engineering. A global bank restructured a twenty-call pipeline into ten small-model calls and ten frontier calls, projected a multi-million dollar revenue impact, won the business case, and then watched the pilot stall in compliance review. Over-caution burns money from the other direction: a financial software company whose agent serves millions of small businesses tuned it so far toward human escalation, out of hallucination fear, that the agent became an expensive routing layer, costing hundreds of thousands of dollars before anyone recalibrated.
4. The build-then-buy cycle
Almost every engineering-led company we spoke with started by building agent evaluation and observability in-house, and almost every one of them is now reconsidering that decision at a predictable point. One fast-growing fintech spent several engineers and multiple quarters building on top of open-source pieces, replacing a vendor product that had grown too slow and too inflexible to keep up. Another team, at a much larger company, tried the same thing internally for three straight quarters and still did not reach what a focused external product delivers out of the box.
The rest of the objection cycle repeats too: integration cost that rivals the cost of building from scratch, and a firm refusal to outsource trust itself. Several platform leads told us, independently, that they would happily buy generic infrastructure (trace storage, orchestration) but will not hand a vendor their quality bar, their access control, or their self-improvement loop. The teams most ready to buy anything here are the ones who already tried to build it and hit the wall. The ones least ready have not felt the pain yet, and nothing anyone says shortens that timeline.
The pattern extends below the application layer. At least three companies we spoke with maintain private forks of the same open-source workflow engine because it breaks under multi-agent state, and a fourth migrated off it entirely after months of workarounds. In each case the maintenance did not disappear; product engineers absorbed it and became an infrastructure team by default.
5. Identity is the least-solved governance problem
If there is one question every serious enterprise conversation eventually arrives at, it is this: who, or what, authorized this agent to act? Agents today are mostly treated as static API routes rather than a first-class identity, with no stable owner, version, or principal attached. One platform team, at a large ride-hailing and delivery marketplace, spent the better part of two quarters building an intermediate identity layer so an agent assumed the correct principal, service account versus user account, before touching anything real.
The fixes converge from different directions. One infrastructure vendor issues short-lived cryptographic identities scoped to a single task, deleted automatically once the task completes. Another enterprise, in regulated financial services, is adapting a cryptographic identity standard originally built for service-to-service authentication and repointing it at agent-to-agent authentication instead. A security-focused platform team described a risk-tiered model: read-only agents get a lightweight approval, agents touching customer data get a compliance review, and anything touching a third-party integration gets a full security review before it ships. The most elegant design we heard never lets the agent hold real credentials at all: policy checks run first, a one-time credential is injected for the single downstream action, and it is destroyed on completion. Meanwhile the mundane version of the problem grinds on, with each new permission an agent needs taking weeks of approval at a large enterprise. Nobody we talked to considers this solved. Several called it the actual reason enterprise AI deployments stall, more than any gap in model capability.
6. Silent regressions, no way to catch them
One story keeps repeating, in different companies, in the same shape: someone changes a prompt, a tool, or a shared memory block, and agent behavior degrades quietly for days before anyone catches it. At one large marketplace company, an update to a shared memory block caused regressions across several unrelated agents. More than a dozen engineers spent the better part of two weeks reconstructing, by hand, what had changed and when. At a customer-support software vendor, a single dropped knowledge base reference caused an agent to hallucinate for days before a customer complaint surfaced it.
The underlying gap is structural, not a one-off mistake. There is no git-equivalent for agent development that ties prompts, tools, memory, and datasets together with a rollback path. Dataset versions are not tagged back to the traces that produced them, so evaluation results regress silently right alongside the agent. Every company burned by this now says some version of the same thing: they need version control for agents, not only for the code that wraps them.
Sprawl feeds the problem, because every duplicate is another surface nobody is watching. At one e-commerce platform, five separate teams had built five implementations of the same search tool, which meant five places for the same regression to hide. At one payments company, a third of everything in the shared agent registry turned out to be redundant, and the fix that worked was organizational rather than technical: core product owners got admin write access, everyone else got read-only. A global bank found its system prompts scattered through application code with no change process at all, until an incident forced one into existence. Velocity makes all of it worse. One engineering manager described a single developer with fifty-five open pull requests and a CI/CD pipeline built for the pre-agent era.
7. What the best teams do differently
The teams who have been at this longest keep landing on the same handful of practices, independently, without comparing notes.
- Boolean or pass/fail judges beat 1-to-5 scores. Two unrelated companies described the identical rule: never use a percentage score, use a deterministic pass or fail instead, because float scores have poor agreement between human raters and hide more than they reveal. The teams most serious about this track judge agreement with human raters using Cohen's kappa and treat a drifting kappa as a ship blocker.
- Golden trace sets anchor regression testing. One financial-services company keeps a rotating set of fifty to a hundred questions its agent must always answer correctly, with a thumbs up or down feedback loop layered on top. Another runs every new data source through an isolated test suite before promoting it to the main product.
- Offline evals only hold up when seeded from live production traffic. Static golden traces go stale within weeks once prices, inventory, or policies shift underneath them. The companies that keep their offline signal honest are the ones re-seeding it from production traces on a real cadence, not once at launch.
- Risk-tiered review beats one-size-fits-all review. Several companies have replaced a single review process with a risk score that routes low-risk changes to a fast lane and anything touching money or customer data to mandatory human review, sometimes backed by an adversarial council of agents that has to reach consensus first.
- Verification, not generation, is the new bottleneck. One platform watched code review time double year over year while generation throughput climbed. One growing software company now routes several thousand merges a week through its internal review agent, most of its pull requests, gated by a single signal: does the finding rise to a real high-severity bug. Nothing else earns a block on its own. The economics of a program like this are now public: DoorDash's writeup of its code review agent reports over ten thousand pull requests reviewed weekly, at a cost of roughly three dollars and a turnaround of about seven minutes per review, with engineers accepting sixty percent of its high and critical findings.
- Annotation throughput is the ceiling on everything else. One team replaced a spreadsheet-based review queue with an IDE-embedded one and watched annotator output roughly triple within a month, with no retraining required. Every judge you calibrate inherits the quality and the quantity of these labels.
8. Why this all comes back to ROI
Every one of these problems is a version of the same gap: the distance between what an agent's trace shows and what the business cares about. Companies know their agents are working, in some rough sense. Almost none of them know with confidence how much better an agent got after a change, why it regressed when it did, or whether the improvement maps to anything a CFO would recognize.
One tell is where eval investment concentrates. The highest-revenue agent anyone described to us sits in the hundreds of millions in annual revenue, and its evaluation team is one of the few teams in the company still hiring. Rigor scales with revenue at risk, and everyone else is still deciding how long they can wait.
That gap is what an evaluation-first, ROI-linked development loop exists to close: seeding evals from production, gating every release through agentic CI/CD, and tying every number back to a business outcome rather than a proxy metric nobody outside engineering believes. We wrote about the methodology in Efficient Agent Evals and the infrastructure behind it in Offline Eval Environments.