AI Compute Gap Widens: Enterprises Outpace Visibility on Infrastructure Costs

Understanding the AI compute gap

In a fresh survey of 107 midsize enterprises, the gap between AI spending and cost visibility is expanding. Companies are snapping up GPU capacity and specialized clouds faster than they can measure the economics, creating a pronounced AI compute gap that threatens efficiency.

Current deployment landscape

Most firms still rely on the big public clouds – Google Cloud (48% usage), Microsoft Azure and AWS – and on model APIs from OpenAI, Anthropic and Gemini. Dedicated AI‑cloud providers such as CoreWeave or Lambda register near zero presence today.

Future buying signals

When asked where they will look next, ↑ 45% say they will evaluate AI‑specialized clouds within twelve months, despite almost none using them now. Another ↓ 83% report GPU utilization at or below 50%, indicating massive idle capacity.

“We are buying on integration and total cost of ownership, not on headline token price,” a CIO noted.

Integration with existing stacks tops the decision criteria (41%), followed by total cost of ownership (35%). Only 8% cite cost per million tokens as decisive.

Switching intent spikes

A striking 64% plan to add or replace an infrastructure provider within a year, and 38% within the next quarter. The moves focus on incumbents – Azure, Google Cloud, OpenAI – rather than newcomers.

Yet fewer than half (44%) can rigorously track compute spend, and 39% only partially monitor costs. Satisfaction scores hover around 4.0/5, with value‑for‑money trailing at 3.9.

Looking ahead, only one in five firms recognize the upcoming inference bottleneck as memory bandwidth, leaving a strategic blind spot.

For a deeper dive, see the full Reuters analysis and related Bloomberg coverage.