Enterprises are in the middle of a paradox that’s starting to look less like a temporary growing pain and more like a structural risk: they’re buying AI infrastructure faster than they can measure what it costs to run.
That’s the core message from a new VentureBeat Pulse Research wave focused on enterprise AI infrastructure, compute utilization, and inference economics. The survey—Q2 2026, with 107 respondents from organizations with more than 100 employees—paints a picture of rapid investment momentum paired with weak visibility into unit economics. In other words, many companies are scaling compute capacity and experimenting with new infrastructure directions while still lacking the instrumentation to understand whether their spend is efficient, predictable, or even controllable.
The result is what the report calls a “compute gap”: the distance between how aggressively enterprises are investing in AI infrastructure and how little of its economics they can see or steer. And the gap isn’t just about cost accounting in the abstract. It shows up in utilization patterns, provider churn intent, and the fact that the next major inference constraint—memory bandwidth and KV-cache capacity—is barely being governed by most teams.
This is not a story about enterprises failing to buy the “right” hardware. It’s a story about enterprises buying the next layer of infrastructure before they’ve built the measurement layer that would tell them what the current layer is costing, how well it’s being used, and what tradeoffs will matter as workloads shift from training-like behavior to large-scale inference.
A maturity mismatch: most are not at “production at scale,” yet budgets are acting like they are
One of the most revealing findings is also the simplest: only about one in five enterprises (21%) say they run AI in production at scale. The majority—76%—are either experimenting or running only some workloads in production.
That matters because “production at scale” is where compute economics become real in a way that experiments rarely require. When workloads are small, teams can often absorb inefficiencies. When workloads are scaled across users, regions, or business processes, inefficiency becomes expensive quickly—and it becomes measurable. But the survey suggests many organizations are still in the earlier stages of deployment while already planning infrastructure evaluations and provider changes that imply a much more mature operating model.
This mismatch creates a dangerous dynamic. If you’re still learning how your workloads behave, you need measurement even more—not less. Yet the report finds that measurement is lagging spending.
The familiar stack today, but the next budget is aimed elsewhere
Right now, most enterprises run AI on a familiar base: hyperscalers and major model-provider APIs. Google Cloud leads at 48% among current usage presence in the stack. General-purpose clouds (including Google, Microsoft, AWS, and Oracle) plus major model APIs (Gemini, OpenAI, Anthropic) account for essentially all current deployment in this cohort.
Specialized GPU cloud providers—the “neocloud” category that has dominated much of the recent AI infrastructure conversation—register at or near zero among these enterprises today. Only 6% run their own on-prem GPU clusters and 4% use a custom open-source stack.
But here’s where the report’s tension becomes sharp. When asked where they plan to evaluate AI infrastructure over the next 12 months, enterprises point away from what they currently use.
AI-specialized clouds are the top planned evaluation area at 45%. That’s not incremental interest; it’s a re-platforming signal. Nearly a third (32%) intend to evaluate non-Nvidia accelerators, and 28% plan to evaluate next-generation Nvidia silicon. Even decentralized compute networks (16%) and sovereign compute (11%) draw meaningful attention.
The direction-of-travel question reinforces the same theme: every infrastructure approach shows net expansion, but specialized AI clouds carry the highest net momentum (+24), edging out even the hyperscalers (+22). In plain terms, enterprises are preparing to move meaningful compute off the general-purpose cloud—even if they haven’t done so yet.
This is also consistent with an earlier VentureBeat Pulse Research wave (April–May) referenced in the report. Back then, usage of specialized AI clouds was similarly marginal (CoreWeave at 3%, Lambda at 4%, Crusoe at 2% of enterprises). Yet when asked what change they planned, moving workloads to specialized AI clouds was the most-cited answer at 33%. Two waves, two different question framings, one consistent pattern: the infrastructure category enterprises are most eager to assess is the one they barely use today.
So what’s driving this? The report doesn’t claim a single cause, but it does offer a clue through the next finding: enterprises say they care about integration and total cost of ownership, not headline token price. That suggests they believe there are economic advantages to specialized compute—but they may not yet have the measurement discipline to prove those advantages in their own environment.
A switching wave is building: churn intent is unusually high
If enterprises were simply evaluating new options slowly, the story might be manageable. But the survey indicates a more aggressive posture.
A clear majority—64%—plan to switch or add an infrastructure provider within 12 months. Even more striking, 38% say they’ll do it within the next quarter.
For a category as foundational as compute, that level of churn intent is unusual. It implies that many organizations are not treating infrastructure selection as a long-term decision with stable economics. Instead, they appear to be treating it as a moving target—one that must be revisited frequently as performance, pricing structures, and architectural constraints evolve.
Where does that switching consideration land? Again, the report points to incumbents. The providers drawing the most switching consideration are Microsoft Azure and Google Cloud (33% each), OpenAI (30%), and Gemini (22%). That suggests much of the near-term movement is reshuffling among the majors and consolidating spend rather than a wholesale defection to new entrants.
Meanwhile, the neocloud interest looks more like a 12-month evaluation thesis than immediate switching. In other words: specialized clouds may be the “next platform” under consideration, while the next quarter’s churn is largely about trading share among existing relationships.
This distinction matters because it implies two different timelines of risk. The short-term risk is operational disruption and cost volatility from frequent provider changes. The longer-term risk is strategic misalignment—moving to a new compute layer without fully understanding the unit economics and bottlenecks that will define inference cost at scale.
Buyers say economics matter—yet measurement is weak
When enterprises choose an AI infrastructure provider, the report finds that headline pricing is not the deciding factor. Integration with the existing stack (41%) and total cost of ownership (35%) dominate. Cost per million tokens is the deciding factor for just 8% of respondents—dead last.
That sounds rational, and it is. Token price is a vendor marketing metric; total cost of ownership is the operational reality. But the report immediately undercuts the confidence implied by that stated priority.
Fewer than half of enterprises (44%) rigorously track the cost and return of their AI compute. The majority either track partially (39%), cannot quantify it yet (20%), or have not prioritized it (6%).
This is the heart of the compute gap. Enterprises are making decisions based on economic criteria they often cannot measure with enough rigor to steer confidently. They may be able to estimate costs at a high level, but the report suggests they lack the instrumentation to connect compute usage, utilization, inference throughput, and business outcomes into a reliable unit-economics model.
And this is where the story becomes more than a survey statistic. If you can’t measure compute costs rigorously, you can’t reliably compare providers on true TCO. You can’t validate whether specialized clouds reduce cost in your workload mix. You can’t detect whether utilization improvements are real or just shifting costs elsewhere. You can’t forecast how costs will behave as inference scales and architectures change.
In short: enterprises may be choosing on economics, but they’re doing it with incomplete visibility.
The compute they already own is often sitting idle
The report also finds that the compute footprint enterprises already have is frequently underutilized.
Among enterprises that operate GPUs, 83% report GPU utilization of 50% or less. Nearly half (49%) run at 25% or below. Only 12% clear the 50% mark, and 8% do not measure utilization at all.
Idle accelerators are expensive accelerators. This is not merely an efficiency issue; it’s a cost-structure issue. If utilization is low, then the effective cost per useful unit of work rises. That makes it harder to justify new infrastructure investments because the baseline is already inefficient.
Yet the survey shows enterprises are planning to evaluate specialized compute categories and switching providers at a high rate. That combination—underutilized existing capacity plus plans for additional infrastructure—suggests that many organizations are treating compute as a capacity problem rather than a measurement and orchestration problem.
It also suggests something else: even if enterprises want to optimize TCO, they may not have the operational telemetry to identify where waste is occurring. Utilization is one metric, but it’s also a proxy for scheduling efficiency, workload shaping, and demand forecasting. Low utilization can mean poor orchestration, unpredictable traffic, or workloads that don’t fit the available capacity patterns. Without measurement, those root causes remain hidden.
The next bottleneck is memory, and many teams aren’t ready
Perhaps the most forward-looking—and potentially most consequential—finding is about the next inference constraint.
As inference scales, the limiting factor often shifts from raw compute to memory bandwidth and KV-cache capacity. The report frames this as a frontier that is not yet governed by most enterprises.
When asked which approach they would rely on as the binding constraint in inference shifts from compute to memory bandwidth, responses scatter across vendors and tooling. Dell leads at 31%, Nvidia follows at 16%, and the rest fragments across storage vendors, open-source tooling, and model-level efficiency techniques.
Most telling is that roughly one in five enterprises (18%) either do not recognize the constraint or have