But this year, the conversation changed. In nearly every executive review, the question is no longer “How do we get more AI?” or “Which model is smartest?” It’s “How do I generate impactful outcomes, while keeping AI costs under control?” Different companies have different answers. Uber tightened internal AI spending after consumption ran far ahead of plan. Anthropic, OpenAI and Google have aggressively cut input token prices and rolled out prompt caching.  And Cursor, with Composer 2.5, now considers cost an important factor in model selection — not just performance. 

Cutting token and model costs are one solution, but they ignore the more profound issue: Most companies are not architected to use tokens efficiently. The modern AI pipeline behaves more like a sieve — leaking tokens, and dollars, at every phase of execution. Pouring cheaper tokens through a leaky foundation isn’t a sustainable solution. To truly generate great, cost-effective outcomes, companies need an architectural foundation that closes those leaks. Here’s how.

Figure 1 — The Leaky AI Pipeline: where tokens, and money, are lost every day.

The Five Pillars Of Token Optimization

Token optimization requires more than smart prompting. It takes an architecture designed to address the five predictable failure points that show up again and again:

1. Context Bloat: Context window bloat is the first leak. The more information you stuff into a prompt, the more you pay and the more the model drifts. The fix comes in adding context before the prompt stage — with a data foundation that agents can rely upon to always provide the most reliable version of the truth. Master data management uses deterministic matching, merging, and deduplication to unify overlapping or conflicting sets of data in one golden record, ensuring that agents always have access to the right customer, the right account, the right policy, the right transaction and the right recent activity — not through the prompt, but built into the infrastructure of every single interaction.
   
2. Ungoverned data access. Without clear governance, agents roam aimlessly through data warehouses looking for credible information, then check the results after the fact — wasting time and tokens, and generating inconsistent actions. Establishing a data catalog, lineage, permissions and quality signals makes governance a routing signal instead of a checkpoint, directing the agent to certified data on the first hop.
   
3. Wrong model. Algorithmic brute force — sending every task to a frontier model — is the most expensive habit in the building. Cost-routing work through the right model, instead of  the biggest one, is one of the best ways to increase efficiency. Cost-route the work. Treat models as a portfolio: small and efficient for lookups and classification, tuned mid-tier for domain workflows, frontier reserved for novel reasoning and sensitive judgment.
   
4. Lack of persistent memory. Every stateless agent starts each interaction from scratch. It has to reload context, reprocess history and rediscover exceptions that were already resolved. Structured memory turns each new interaction into an incremental one. The agent keeps and retrieves the useful state when needed — relevant facts, prior decisions, user preferences and open tasks — instead of starting from zero. That reduces token consumption, improves consistency, and lowers cost per interaction.
   
5. Single-use semantics. Too many enterprise agents process recurring requests as if they are seeing them for the first time. For example, re-reading the same context, re-running the same retrieval, and re-generating answers that could have been cached, embedded or pre-computed. By pairing prompt caching, embedding reuse and pre-computed outputs with the semantic structure already present in enterprise data flows, companies can make repeat requests progressively cheaper, faster and more reliable

Token Efficiency Is The Next Competitive Advantage

Token optimization is the difference between inefficient experiments and productivity-driving foundations of the business. 

A large enterprise running AI agents for sales or service operations might process thousands of agent interactions daily. Each interaction that passes raw, uncurated data into an LLM prompt consumes 5–10x more tokens than necessary. At $10–15 per million tokens, that compounds fast across a fleet of agents. Enterprises that invest in Master Data Management and data governance create a natural advantage — drawing on pre-curated, semantically enriched, quality-scored data rather than raw noise. 

I’ve seen this advantage firsthand with the customers I work with every day. I see this firsthand in my work with enterprise customers. When Informatica’s master data management integrates with Data 360, Salesforce’s Customer Data Platform, it ensures every AI agent is grounded in trusted customer and business context.  Together, the  architecture turns un-governed AI consumption into governed, measurable business value. 

Every company can buy more tokens. Very few know how to extract more value from fewer of them. The next phase of enterprise AI will not be defined by who consumes the most tokens, runs the largest models, or fills the biggest context windows. It will be defined by who can turn AI consumption into governed, measurable business value.

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.