AI’s new cost equation: Why token economics matters
Executive Take
Token costs will quietly become a line item every CTO has to defend to the CFO. Companies that match model size to task and cache aggressively will spend a fraction of what competitors do.
Executive Summary
At YourStory's DevSparks Hyderabad 2026, NVIDIA's Jigar Halani said AI token consumption is rising fast, NVIDIA budgeted 16 trillion tokens for 2026 but crossed 18 trillion by August, excluding Microsoft Copilot use. He outlined a token economy of utility, demand, supply, and monetization, urging model selection by task and caching to cut costs.
Why It Matters
Technology leaders need to treat AI token spend like cloud spend before it spirals, since agentic AI multiplies tokens through reasoning loops and tool calls. CIOs and CTOs should build cost models now, not after the bill arrives.
Bizquad Perspective
Most leaders are optimizing token cost when the real prize Halani names, training models to get smarter per token and locking in customer stickiness, is the actual profit lever.