AI IN PRODUCTION

You're Probably Paying 5x Too Much for Your AI Calls

Published May 22, 2026 — 3 min read

TL;DR: Most enterprise teams default to the biggest model available — and bleed budget on problems that a much cheaper model handles just as well. The gap between Claude Sonnet 4.6 and Opus 4.7 in real production workloads is under 2 percentage points on benchmarks, but the cost difference is nearly 5x.

Key Insight

"Bigger model" has become a security blanket, not an engineering decision.

Here's the actual math: a typical request at 2,000 input + 500 output tokens costs ~$0.068 with Opus 4.7 versus ~$0.014 with Sonnet 4.6. Run 1M of those calls a month — a modest enterprise volume — and you're looking at $68,000 vs. $14,000. Same task. Same output quality in most cases.

The uncomfortable question: did your team ever actually benchmark both, or did someone just pick Opus because it "sounds better for important work"?

On SWE-bench Verified and OSWorld-Verified — two of the most demanding agentic coding benchmarks — Sonnet 4.6 trails Opus 4.7 by less than two percentage points. For the workloads most enterprise teams are actually running (code generation, summarization, document analysis, customer-facing chatbots, CRUD-adjacent automation), that gap disappears entirely in practice.

Why Teams Miss This

The mistake is treating model selection as a one-time decision at project kickstart, usually made by whoever wrote the first proof-of-concept. They used Opus, it worked, and now it's baked into every subsequent deployment.

No one goes back. Why would they? It's working. The cost feels like an infrastructure line item, not a product decision.

The second failure: treating model choice as binary (Opus or Haiku) instead of routing dynamically based on task complexity. Teams that build routing logic — "send routine tasks to Sonnet, escalate ambiguous or high-stakes reasoning to Opus" — consistently land at 60–80% cost reduction with zero degradation in user-facing quality.

The third failure: benchmarks. Most teams evaluate model quality against their internal vibe ("that response felt better") instead of structured evals tied to their actual task distribution. If you don't know what percentage of your requests are "hard," you can't make a rational model choice.

How to Actually Do It

Step 1: Audit your current task mix.

Pull a random sample of 200 production requests. Classify each as routine (instruction-following, retrieval, summarization, boilerplate gen) vs. complex (multi-step reasoning, ambiguous context, adversarial inputs). If >70% are routine, you have a strong case to route them to Sonnet.

Step 2: Run a shadow eval on Sonnet.

For your top 3 task types, run the same prompt batch against both Sonnet 4.6 and Opus 4.7. Score outputs on your actual quality criteria, not vibes. Most teams find <5% difference on routine tasks.

Step 3: Implement model routing.

def route_model(task_type: str, complexity_score: float) -> str:

# Escalate to Opus only when genuinely needed

if complexity_score > 0.8 or task_type in HIGH_STAKES_TASKS:

return "claude-opus-4-7"

return "claude-sonnet-4-6" # Default for the vast majority

Step 4: Add prompt caching.

If you're not caching shared context (system prompts, long documents, few-shot examples), you're paying full price on every call. Anthropic's prompt caching drops repeated input token costs by up to 90%. On a long system prompt reused across thousands of calls, this alone can cut your bill in half — independent of model choice.

Step 5: Measure and iterate.

Track model-per-task-type alongside your existing quality metrics. Set a threshold: if output quality drops more than X% on Sonnet, escalate that task type to Opus permanently.

What We've Learned

The best AI cost optimization move for most enterprise teams isn't better compression or rate-limit tricks. It's just actually making the model selection decision instead of defaulting to the flagship.

Audit your task mix this week. Run the shadow eval. If the numbers hold — and they usually do — implement routing before your next billing cycle. Teams doing this consistently land at 60–75% cost reduction with no measurable quality regression for end users.

The next experiment worth running: automated complexity scoring at inference time, so routing happens dynamically without human classification. A lightweight classifier (or even a rule-based system on token budget and task metadata) can make this decision in <5ms.

Sources