AI Firms Struggle With Unpredictable LLM Token Costs

Businesses implementing Large Language Models and agentic AI are struggling to manage pricing and budgets due to the unpredictable nature of token consumption. While the cost per individual token has decreased, the total volume of tokens used by enterprises is growing rapidly. Goldman Sachs forecasts that monthly token consumption will increase 24-fold, reaching 120 quadrillion by 2030.

This financial unpredictability is driven by variations in user prompts and the complex internal interactions required by agentic systems. These volatile costs have led to immediate budget failures; for example, Uber Technologies Inc exhausted its annual AI coding budget within a few months. In response to rising expenses, Microsoft has reportedly restricted its engineers from using certain third-party coding tools.

Industry leaders suggest that current financial planning for AI is largely speculative. Simon Gooch of the identity management company Saviynt stated that long-term cost models for AI currently lack a logical basis due to prevailing economic uncertainty.