AgentOps: The Observability Stack That Keeps AI Agents Out of Trouble

57% of companies have AI agents running in production. Most can't tell you what those agents did yesterday — what tools they called, what data they touched, or why a run cost three times more than expected. That's not an agent problem. It's an observability problem.

Why "Ship and Hope" Stopped Working

The jump from AI demo to production agent sounds straightforward. It rarely is. G2's 2025 AI Agents Insights report found that 57% of companies already have AI agents in production — but anecdotally, the failure stories are everywhere. Runaway costs. Agents that hallucinate tool parameters. Loops that retry indefinitely because nobody defined a stopping condition.

The underlying problem is almost always the same: teams treat agents like software they deploy and forget. But agents aren't deterministic processes. They're probabilistic reasoners that make decisions at runtime — and those decisions change when the model updates, when context shifts, or when an upstream API starts returning unexpected data.

Wang et al. (2025) formalized this challenge in their survey "A Survey on AgentOps", proposing a four-stage operational framework — monitoring, anomaly detection, root cause analysis, and resolution — specifically built for LLM-powered systems. It's the most useful framing I've seen for operationalizing agents at scale, and it maps cleanly to tooling that exists today.

What AgentOps Actually Covers

AgentOps borrows from DevOps and MLOps, but it has its own distinct concerns. Here's how they break down in practice:

1. Distributed Tracing

Every agent run is a tree of decisions. An agent receives a task, reasons about it, calls a tool, receives output, reasons again, calls another tool. Each of these steps — called a span — should be captured and linked to the parent trace. Without this, debugging is archaeology: you know something went wrong, but you can't reconstruct why.

Arize Phoenix and Langfuse both use OpenTelemetry-based instrumentation for this. You get sessions → traces → spans → LLM calls, captured end-to-end. LangSmith does something similar with a tighter LangChain integration. The important thing isn't which tool you pick — it's that every production agent run is traceable by default, not just when something breaks.

2. Cost Baselining and Anomaly Alerting

Token cost is the most underappreciated operational metric in agentic AI. A single misbehaving retry loop can run up hundreds of dollars in an afternoon. Modern observability platforms track cost per trace in real time — and more importantly, can alert when a run exceeds a configured threshold.

The right baseline isn't a flat dollar limit. It's a per-task cost envelope: what should this agent spend, on average, to complete this type of task? If a lead enrichment agent normally costs $0.04/run and you're seeing $0.40 runs, that's a signal — either the task scope changed, the model is struggling, or something upstream broke and the agent is retrying into a dead end.

3. Behavioral Drift Detection

This one is subtle but critical. Agents drift — not because you changed anything, but because the model provider silently updated the underlying weights, a dependency API changed its response format, or input data characteristics shifted over weeks.

Galileo's Signals engine automates failure mode analysis by scanning production traces and identifying drift patterns — then prescribing specific fixes for prompt engineering or retrieval strategies. Arize's own platform does similar work through ML monitoring principles applied to LLM outputs. The manual version of this is running a fixed eval suite weekly against your production agent and charting whether scores hold.

4. Human-in-the-Loop Escalation

Observability surfaces problems. Escalation handles them. UiPath's AgentOps guide frames this clearly: every production agent needs a defined escalation model for high-impact actions and exceptions. What triggers a human review? Who gets notified? What's the SLA for a human response before the agent times out?

This isn't just about safety — it's about trust. Teams that can show auditors a clean trace of every action an agent took, plus a log of every escalation and approval, have a dramatically easier time expanding agent scope over time.

The Tooling Landscape (Practical Tier List)

There's no shortage of platforms competing in this space. Here's how I'd categorize them for teams at different stages:

The common thread: all of them expose spans via OpenTelemetry, which means you can forward traces to your existing monitoring stack (New Relic, Datadog, Snowflake) without a full migration. Pick based on your framework, your self-hosting appetite, and whether you need evals baked in or separate.

An AgentOps Checklist You Can Actually Use

Adapted from UiPath's enterprise AgentOps checklist and the Camunda agentic orchestration guide, these are the questions every team should be able to answer before putting an agent into production:

• Ownership: Do we know what each agent is responsible for, and who owns it when it breaks?
• Tool control: Can we list exactly which tools the agent is allowed to call, and with which inputs?
• Auditability: Can we reproduce what the agent did on any given run — tool call sequence, data accessed, decisions made?
• Pre-release validation: Can we validate tool choice and execution path before release, not just outcome?
• Drift detection: Do we have evals running on a schedule that would catch regression before users notice?
• Cost forecasting: Can we bound and project cost by task type, model, and context size?
• Rollback path: Do we have version control and environment promotion for agent configs and prompts?
• Escalation model: Is there a clear, tested human-in-the-loop process for high-stakes actions?

If you can't answer "yes" to all eight, you're not running production agents — you're running experiments in production. The difference matters more than most teams realize until something goes wrong.

The Sequencing That Actually Works

Most teams build observability as an afterthought, after they've already shipped the agent. That's backward. The right build order:

1. Instrument first. Add tracing before you write agent logic. Every LLM call, every tool invocation, every decision branch should emit a span from day one.
2. Establish your baseline. Run your agent against a fixed eval set before going to production. That's your regression anchor.
3. Set cost and latency thresholds. Know your P50 and P95 run cost and time before you see production traffic. Alerts are useless without a baseline.
4. Define the escalation path. Who gets paged when the agent hits a confidence threshold that requires human review? Document it before the agent ships.
5. Run evals on a schedule. Weekly minimum. More often if you're on a model provider that does frequent releases.

Nikola Balić's Agentic AI Handbook captures this well: "The hard part isn't getting a demo — it's making the loop reliable." Reliability doesn't come from the model. It comes from the operational scaffolding around it.

The Bottom Line

AgentOps isn't a vendor category — it's a discipline. The tooling is maturing fast (Langfuse, Arize, Galileo, Braintrust, LangSmith are all legitimate options in 2026), but the discipline has to come from the team building the system. That means: trace everything, baseline costs before you have incidents, run evals on a schedule, and design your escalation path before the agent ships. For a comprehensive checklist of what that operational scaffolding should look like, see our agent ops runbook.

Teams that do this can expand agent scope confidently, because they have evidence. Teams that skip it spend every week firefighting something they can't fully explain. The gap between those two groups will define which organizations actually benefit from agentic AI — and which ones quietly deprecate their agents and go back to scripts.