AI, Actually

Tokens Are Why Your AI Bill Looks the Way It Does

Published August 11, 2026 — 4 min read

TL;DR: Every AI product you buy — whether it's a $20/month consumer tool or a six-figure enterprise contract — is built on token math that the vendor rarely explains. If you're signing off on AI spend without understanding tokens, you're approving a variable-rate mortgage without reading the terms.

Key Insight

Tokens are the unit of measure behind every AI pricing tier, every context limit, and every "why did this cost so much" conversation you'll have with your team or your CFO.

Vendors don't say this plainly, but you're paying for processed text, not AI, at rates that vary by 5-50x depending on which model you chose and whether your prompts are written efficiently.

A rough rule: one token ≈ four characters of English text. "The quick brown fox" is five tokens. A 10-page legal brief is roughly 2,500 tokens. A full 200-page contract plus a detailed system prompt and the model's response is easily 150,000 tokens — and that's one interaction.

Why Teams Miss This

Two failure modes show up constantly in enterprise AI deployments:

Failure Mode 1: Buying by seat, budgeting by vibes.

Most SaaS AI tools are priced per seat per month, which hides the token math entirely. But under the hood, that tool has a cost ceiling — either a hard context limit (the model can only "see" so many tokens at once) or a soft one baked into the product tier. Teams hit the wall mid-project and don't know why the output suddenly got worse or the feature stopped working.

Failure Mode 2: Defaulting to the biggest model.

The assumption is: bigger model = better results = worth it. Sometimes that's true. Often it isn't. If you're running a customer FAQ classifier that needs to pick from 12 options, you do not need the same model you'd use to synthesize a 300-page RFP. The cost difference is not small.

Current API pricing (as of August 2026):

Running 1,000 FAQ classifications per day at Haiku rates costs about $0.30/day. Running the same workload on Fable costs $3-15/day — 10-50x more, for a task where Haiku wins on accuracy too, because the task is simple and the smaller model isn't distracted by capability it doesn't need.

How to Actually Do It

Step 1: Audit what you're actually sending.

Before you optimize, measure. If you're calling an API directly, log your input_tokens and output_tokens from the response metadata. Most APIs return this per call. Sum it weekly.

import anthropic

client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-sonnet-5-20260601",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Classify this support ticket: ..."}]
)

# This is your billing meter
print(f"Input: {response.usage.input_tokens} tokens")
print(f"Output: {response.usage.output_tokens} tokens")
print(f"Approx cost: ${(response.usage.input_tokens * 2 + response.usage.output_tokens * 10) / 1_000_000:.4f}")

Step 2: Right-size the model to the task.

Define your task categories — classification, summarization, generation, reasoning — and test the smallest model that hits your quality bar. Most classification and extraction tasks run fine on Haiku-tier models. Synthesis and judgment tasks may need Sonnet or Opus.

Step 3: Watch your context window, not just your output.

The context window is the maximum number of tokens a model can hold in memory for one interaction: your system prompt + conversation history + the document you pasted + the response. For long workflows, the input tokens often dwarf the output. A 128k-token context window means roughly 90-100 pages of text. If your workflow routinely fills it, you're paying for that every single call.

Step 4: Use prompt caching for repeated content.

If you send the same system prompt or document chunk in every call, most APIs now offer prompt caching. Anthropic's caching cuts the cost of repeated input to $0.50/MTok (vs $5 for fresh Opus 5 input). For high-volume workflows with stable system prompts, this is a 90% input cost reduction. It requires explicit cache control headers. It's not automatic.

Step 5: Set hard budget caps before you scale.

Every major AI API supports spend limits and usage alerts. Set them before you run your first production workflow, not after you see the bill. A runaway agent loop can burn through a monthly budget in an afternoon.

What We've Learned

The single highest-impact habit here is printing the token count next to every AI output during development, in your dev/test environment where your team actually sees it, not buried in production logs. Once engineers see "this prompt costs 12,000 tokens and the next one costs 800," they start writing better prompts without being told to. Token awareness is a craft skill that compounds.

The next experiment to run: take your three most expensive AI workflows, swap the model one tier down, and run 50 test cases through both. The quality delta is usually smaller than you expect. The cost delta is usually larger than you realized.

Sources

Have a specific workflow in mind?

Bring it to a Quick Scan — a live working session where we'll tell you honestly whether it should be an agent, a workflow, or left alone, before you spend a dollar building it. You get 3 prioritized recommendations on the call, a one-page summary after, and the $500 credited toward any engagement within 30 days.

See AI Agent Consulting →·Book an intro call →

Get new posts + practical agent-ops notes

One email when something new goes up. No nurture sequence, no spam — unsubscribe whenever you want.

Thanks — you're on the list.