Your Central AI Team Is the Bottleneck, Not the Engine

Key Insight

The centralized "AI factory" feels like the responsible choice. One team, one platform, one set of standards. The trouble is that the hard part of an AI feature is rarely the model call. It is knowing what a good answer looks like for claims intake, or for a supplier dispute, or for a field technician at 6 a.m. That knowledge lives in the business unit, and a central team can only get it through meetings.

So every project becomes a translation exercise. The business unit writes a request, the central team interprets it, builds something, and returns it for review. Each loop costs weeks. Evaluation is where it hurts most, because the people who can judge the output are not the people tuning the prompts.

Teams that embed ownership flip this. A product owner in the business unit owns the use case, the eval set, and the outcome metric. Engineers who sit next to that owner ship changes in days. The central team stops being a factory and becomes a small enabling team that provides the paved road: approved model access, logging, a shared eval suite, security review templates, and cost visibility.

This is the same lesson platform teams learned about infrastructure, and the same one behind data mesh: Team Topologies describes enabling teams and platform teams as support for stream-aligned teams, not replacements for them. AI is following the same arc, only faster, because prompts and agents are cheap to change and expensive to get wrong.

Why Teams Miss This

Three assumptions keep the factory model alive.

• Scarcity of talent. Leaders assume AI skills are too rare to spread out, so they pool them. Pooling does concentrate skill, but it also puts every request behind one prioritization meeting.
• Governance anxiety. Central control looks safer. In practice, a backlog pushes business units to buy their own tools quietly, and you get shadow AI with no logging at all.
• Demo success. The first two projects usually go fine because the central team had spare capacity. Nobody measures what happens at project twelve, when the queue is five months long.

McKinsey's State of AI survey has repeatedly found that organizations that report measurable returns tend to redesign workflows and assign senior ownership for adoption, rather than treating AI as a technology rollout. The tooling is similar across companies. The ownership model is where they differ.

How to Actually Do It

Start by splitting responsibilities into what must be shared and what must be local.

Shared (the enabling team owns it):

• Model gateway with approved providers, key management, and spend caps
• Logging and trace storage so every call is auditable
• A reusable eval suite and a template for writing golden test sets
• Security and legal review checklists, pre-answered for common patterns
• Office hours and embedded pairing for the first project in a new unit

Local (the business unit owns it):

• The use case backlog and its priority
• The eval set and the definition of "good"
• Prompt, agent, and workflow changes
• The outcome metric that someone's bonus actually depends on
• On-call for the thing they shipped

Then make the contract explicit. A short file in each unit's repo keeps it honest:

# ai-ownership.yaml
unit: claims-intake
owner: jane.doe@example.com
outcome_metric: first_touch_resolution_rate
eval_set: evals/claims_v3.jsonl
min_eval_pass_rate: 0.92
gateway: shared-llm-gateway
review_cadence_days: 14
escalation: ai-enablement@example.com

The enabling team's job is to make this file easy to fill in and hard to skip. Its own success metric should be time from first idea to first production call in a new unit, not the number of projects it personally delivered. If that number is not falling quarter over quarter, the team is drifting back into being a factory.

Keep the enabling team small. A rough rule that holds up: if it has more than one engineer for every three or four business units it supports, it is probably shipping product work instead of building the road.

What We've Learned

The experiment worth running this quarter is simple. Pick the business unit with the longest AI backlog and measure its request-to-production time today. Then give that unit a named owner, an eval set they wrote themselves, and access to the shared gateway. Leave the central team in a pure support role for 60 days.

If cycle time drops and quality holds on the unit's own evals, you have your evidence for the next unit. If quality drops, you have learned that the shared eval suite is too thin, which is useful and cheap to fix. Either way you stop arguing about org charts in the abstract.

The queue is slowing you down, not the models.

Sources

• Team Topologies, key concepts on stream-aligned, platform, and enabling teams: teamtopologies.com/key-concepts
• McKinsey QuantumBlack, The State of AI survey series: mckinsey.com state of AI
• Martin Fowler, Data Mesh Principles and Logical Architecture, on domain ownership of data products: martinfowler.com/articles/data-mesh-principles.html
• Latent Space, AI engineering newsletter and podcast (topic inspiration): latent.space