The Cost of (Enterprise) AI explained & How to Manage

Welcome to Between the Hype. A biweekly newsletter on where enterprise systems and AI actually intersect. Not the keynote version. The version you need when you’re back at your desk on Monday.

The Fundamentals — What you’re paying for: tokens

Large language models read and write in tokens: chunks of text, roughly three-quarters of a word each. Every model is priced per token, a set rate for what you put in and another for what it generates. A one-line question is a few hundred tokens and costs a fraction of a cent. That’s why “just try it” feels free.

The bill changes shape the moment AI stops answering and starts working. An autonomous agent doesn’t take one turn. It reads documents, reasons across several steps, calls tools, and checks its own output, and each of those is more tokens, often thousands of them, for a single task. The unit price didn’t move; the number of units exploded. That’s where the real cost of enterprise AI lives: in consumption, not in the licence on the contract.

How Microsoft, SAP & Oracle meter it

The big enterprise vendors wrap that token in their own currencies, but it’s always underneath.

Microsoft is the most explicit. Copilot activity is billed in “Copilot Credits,” a metered currency drawn from a tenant-level pool you top up with prepaid Capacity Packs (around $200 a month for 25,000 credits) or pay-as-you-go at about a cent per credit. The rates tell the story: a generated answer costs roughly 2 credits, an autonomous agent action about 5, and that’s per step, while a premium reasoning model costs far more again. A short multi-step agent run can burn ten times what a single prompt does, every time it fires.

SAP splits it in two. Joule’s base assistant (navigation and simple transactional help) is included in the cloud licence at no extra cost. The premium and agentic capabilities are priced separately, through a virtual currency SAP calls “AI Units,” billed either per user per month or by consumption, depending on the feature. So the copilot is free; the autonomous, high-volume work is the metered part.

Oracle keeps more of it inside the licence. Its pre-built Fusion AI agents come included with the SaaS subscription, and the AI Agent Studio used to configure them carries no additional charge. The meter shows up at the edges: building custom agents needs a paid subscription, premium-model use is charged once you pass a default token allocation, and pooled “Fusion AI Units” consumption starts counting from the 26C release.

Different mechanics, same shape: the assistant is bundled, the autonomous use is metered. Two companies can buy identical licences and end the quarter with very different bills, and very different value to show for them. The difference isn’t the contract. It’s behaviour. And the variable is your people.

Most rollouts teach people how to use Copilot or Joule: here’s the button, here’s a prompt, here’s what it can do. Almost none teach the second half: when it’s worth it. When a one-line prompt does the job, and when you actually need the agent. What a workflow costs versus what an answer costs. That gap is where AI budgets bleed, and just as often where the value underperforms, because the other failure mode is people avoiding the tool entirely since they don’t trust what it might cost.

How to manage it

Cost-awareness is getting more attention; “FinOps for AI” is becoming its own discipline, and finance leaders are starting to ask where the consumption goes. That’s the right instinct, because it’s trainable.

Put the meter on a dashboard. Consumption is already visible in the admin tools — Copilot Studio and the Power Platform admin centre show the credit draw; SAP shows AI Unit consumption. Pull it per team, review it monthly, and set an alert at around 80% of the pool. You can’t manage a number nobody looks at.

Estimate before you scale, not after. Microsoft ships a usage estimator for agent credits; use it (and the equivalent on your stack) to model what a pilot costs at full production volume before you roll it out. Treat the pilot’s usage as a floor, not a ceiling.

Teach the right tool for the job. Most value sits in the cheap, built-in assist, not the expensive autonomous agent. Summarising a thread is a near-free inline action; a multi-step reconciliation is an agent run that costs many times more.

People can only choose well if they know the menu, which is half the reason I keep a running, plain-English map of what each embedded AI feature actually does. Knowing the built-in option exists is what stops someone reaching for an agent to do a job a standard function does for a fraction of the cost.

Give each team a budget it owns. Split the tenant pool, hand each team its number, and let it feel the trade-off. FinOps proved years ago that ownership changes behaviour faster than any policy.

Hand people one rule they’ll remember. Before firing an agent, ask whether the task needs a workflow or just an answer. Most don’t need the workflow.

Between the Hype goes out every other week, on where enterprise systems and AI actually intersect. If it was useful, forward it to whoever owns your AI rollout.

Dive deeper: platform-by-platform guides

Everything mentioned above — the embedded capabilities, the activation steps, the honest limitations — is covered in detail in my practitioner guides. Each one walks through what’s actually available today, how to enable it, and where the gaps are.

Executive overview: the strategic decisions that matter →