ENTERPRISE AI

Stop Using Your Frontier Model as a Workhorse

Published May 19, 2026 — 3 min read

TL;DR: Most enterprise AI teams are routing every request — from simple classification to complex reasoning — through their most expensive model. That's not safe, it's wasteful: teams overspend 50–90% of their LLM budget and call it "quality insurance."

Key Insight

Your AI infrastructure is a fleet, not a single vehicle. Treating GPT-4 or Claude Opus as your default model for everything is like a logistics company that only sends semi-trucks for local deliveries because "bigger is safer." It's not safer — it's just expensive and slow.

Here's the uncomfortable math: premium frontier models (GPT-4, Claude Opus) run $30–60 per million tokens. Small models run $0.10–$0.50 per million tokens. That's a 60–300x price difference. Yet most enterprise pipelines don't route at all — they have one model, one endpoint, and a growing invoice.

Research from UC Berkeley and Canva's production deployments shows intelligent routing delivers 85% cost reduction while maintaining 95% of frontier model performance. That's not a trade-off. That's a bad default corrected.

Why Teams Miss This

The default is conservative for understandable reasons: teams pick one flagship model, it works in the pilot, and then nobody wants to be the person who "downgraded" the AI and broke something.

But this creates a category error. Not all LLM calls are the same:

The model-routing conversation never happens because cost isn't visible until the bill arrives. By then, the architecture is baked in. According to LeanLM's 2026 analysis, enterprises routinely overspend 50–90% of their inference budget — not because they need that power, but because they never designed a routing layer.

There's also a latency cost nobody talks about: large models are slower. If you're building customer-facing features, routing a simple intent-classification call through a frontier model adds 800ms–2s that a small model handles in under 200ms.

How to Actually Do It

Model routing doesn't have to be complex. Start with a two-tier system:

Tier 1 — Lightweight (default): Handle 70–80% of your calls here. Classification, extraction, summarization of short text, intent detection. Use: `claude-haiku-4-5`, `llama-3.1-8b`, `mistral-7b`.

Tier 2 — Frontier (escalate): Reserve for multi-step reasoning, ambiguous inputs, high-stakes decisions with downstream consequences. Use: `claude-opus-4-7`, `gpt-4o`.

Routing logic — three rules:

def route_request(task_type: str, input_length: int, stakes: str) -> str:

# Rule 1: Short structured tasks → lightweight

if task_type in ["classify", "extract", "summarize"] and input_length < 2000:

return "haiku"

# Rule 2: High-stakes or ambiguous → frontier

if stakes == "high" or task_type in ["reason", "plan", "synthesize"]:

return "opus"

# Rule 3: Default mid-tier for everything else

return "sonnet"

This is naive but effective as a starting point. Tools like LiteLLM, Portkey, and Requesty add semantic similarity routing, performance-based fallbacks, and cost dashboards on top of this logic — worth the integration if you're past 10M tokens/month.

Measure before you optimize. Before building a router, log model calls for two weeks with task type and token count. You'll almost certainly find 60–70% of calls are simple enough for a lightweight model. That's your routing target.

What We've Learned

The teams that get this right share one habit: they instrument their LLM calls from day one. Task type, token count, latency, model used, and output quality score. Without that data, routing is guesswork.

If you're running production AI today, pull your last 30 days of API logs and classify each call type. If more than half are classification, extraction, or short summarization tasks — you have an immediate cost-reduction project that requires zero model fine-tuning and zero quality trade-off.

Start there. The frontier models can wait.

Sources