Blog/AI Engineering

AI Agent Observability: What to Log and Alert On

When an agent does something unexpected, can you explain it? Here is what to log per run, which seven signals to watch, which alerts deserve a page and the incident routine to agree first.

17 Nov 20268 min readByteandRise team
seo lead · briefwriter · drafteditor · passpublished

Someone in finance or support asks why the agent refunded a customer last Tuesday, and nobody can answer. The logs show an HTTP 200 and a final message. The reasoning, the tools it called and the document it read are gone. That gap has a cost: hours of engineers reconstructing events, a customer who waits, and a business that stops trusting the agent. AI agent observability closes it by recording enough about every run that you can explain, replay and fix what the agent did.

This guide covers what to record per run, what to watch, which alerts deserve a page, and the incident routine to agree before the first bad run.

Why ordinary monitoring misses agent failures

A normal service fails loudly. It throws, times out or returns a 500. An agent mostly fails quietly: it returns a fluent answer that is wrong, picks the wrong tool, loops until a budget runs out, or drifts after a provider updates a model.

Uptime and latency dashboards stay green through all of that. AI agent observability has to answer different questions:

  • Did the run reach the right outcome, not just finish?
  • Which steps, tools and documents produced it?
  • What did it cost, and who approved it?
  • Is behavior changing week over week?

The tracing vocabulary already exists. OpenTelemetry publishes semantic conventions for generative AI, covering spans for model calls and agent operations. Using a standard means your traces can go into the backend your platform team already runs, instead of a separate silo.

AI agent observability starts with a minimum useful trace per run

For AI agent observability, treat one agent run like one request with a tree of child steps. Give it a run ID and propagate that ID through every model call, tool call and handoff. Without a shared ID you have logs, not a trace.

Field Why you need it
Run ID, parent run ID, trigger Join steps, and find the event or person that started the run
Agent version: prompt, model, tool set Explain behavior changes after a release or provider update
Input reference What the agent was asked, stored by reference where possible
Each model call: model, tokens in/out, cached tokens, latency Cost and latency per step
Each tool call: name, arguments, result status, retries Tool errors are a frequent source of agent failures
Decision points: options considered, choice, stated reason The answer to "why did it do that"
Gate results: which checks ran, passed or failed Ties to your quality gates
Output and destination What left the system, and where it went
Human review: who, when, approved, edited or rejected The audit trail for any decision with consequences
Total cost and end status Feeds cost control and reporting

This is the audit-trail layer of a production agent harness. If the harness owns tool calls and handoffs, it can emit these records without each agent author remembering to.

Keep sensitive data out of the logs

A full trace can contain customer names, account numbers and contract text. Decide up front what is stored:

  1. Store references, not payloads. Log a document ID and a hash, and keep the content in the system of record with its existing access controls.
  2. Redact at the edge. Mask identifiers before the trace leaves the agent process, not in the dashboard.
  3. Split access. Engineers see structure and metadata. Raw content needs a named role and is itself logged.
  4. Set retention per field. Metadata can live longer than content. Ask your legal team which retention rules apply; we do not give legal advice.
  5. Keep the store where your data lives. If the agent runs in your cloud account, its traces should stay there too.

What to watch: seven signals

Record many fields, but chart a few. For AI agent observability, these seven signals cover the failure modes we plan for.

Signal What it tells you Typical failure it catches
Quality score on sampled runs Whether outputs still meet your criteria Silent drift after a model or prompt change
Tool failure and retry rate, per tool Whether integrations are healthy An upstream API changed its response
Steps per run Whether the agent is wandering Loops and repeated retries
Cost per completed task Whether spend tracks value A prompt change that doubled context size
Escalation rate How often the agent hands off to a person Too low can mean overconfidence, too high means it adds little
Human override rate How often reviewers edit or reject output The closest live proxy for wrong answers
Gate failure rate, per gate Which check fires and how often A new input type the gates never saw

None of these has a universal threshold. Set baselines from the first weeks of pilot data, then alert on movement from that baseline rather than on a number copied from a blog post, including this one.

Sample production runs against your evaluation set

Good AI agent observability includes sampling. Your pre-release evaluation set tells you the agent was good on the day you shipped. Production tells you whether it still is. Score a sample of live runs with the same criteria, using automated checks where the answer is checkable and human review where it is not. Add every real failure to the evaluation set so the same bug cannot return unnoticed.

How we run ByteandRise

This post went through the same gate before you read it.

See the whole run →

Which alerts deserve a page

Alert fatigue undermines AI agent observability faster than missing data. Split signals into three tiers.

Tier Examples Response
Page now Spend cap hit, a run touching a system it should not, a tool failing across all runs, a gate bypassed Pause the agent, start the incident routine
Ticket today Tool error rate above baseline, a spike in steps per run, a new failure category in sampled runs Owner investigates within the working day
Weekly review Quality score trend, override rate, cost per task, escalation rate Reviewed with the business owner, no alert

Page only when waiting makes things worse: money moves, data leaks or the agent acts outside its scope. Everything about trends belongs in a review, not on a phone at night.

Pair alerts with controls that act without a human. A hard spend limit per run, a step limit and a kill switch per agent should stop damage before anyone reads the alert.

An incident routine for agents

Agree this before launch. The steps follow the usual incident pattern, adapted for non-deterministic systems.

  1. Pause. Stop the affected agent or route its work to a human queue. A kill switch should take one action, not a deploy.
  2. Contain. List runs in the affected window. The trace tells you which customers, records or systems were touched.
  3. Trace. Open a failing run and find the first step that went wrong: a bad retrieval, a tool result misread, a prompt that changed.
  4. Fix. Change the smallest thing that explains the failure: a prompt, a tool schema, a gate, a model version.
  5. Re-run. Replay the failing inputs, then the full evaluation set. Add the failing case to it permanently.
  6. Resume gradually. Restore traffic in steps, with a person reviewing the first runs.
  7. Write it up. Record cause, impact, fix and the signal that should have caught it earlier.

Name an owner for each step. Usually the business owner of the workflow decides when to pause and resume, and engineering owns the trace and the fix. If nobody owns the pause decision, it will not happen at 2 a.m.

The monthly report the business should receive

Operations and finance leaders do not need traces. They need a one-page summary from the same data:

  • Volume handled, and share completed without a person
  • Cost per completed task, against last month
  • Quality score on sampled runs, and override rate
  • Incidents: what happened, cause, fix, time to pause
  • Changes shipped: prompts, models, tools
  • Open risks and what you plan to change next month

Observability data should serve the business too. This report is also what you hand to a risk or audit team when they ask how the agent is supervised.

How ByteandRise approaches AI agent observability for your company

AI agent observability is cheapest when designed into the first build. This is how we would stage it.

  1. Scoping sprint. We review one workflow, the systems the agent would touch and your data rules. You get a written list of what to log, where it is stored, who can see it, plus the risk and human-review points, as part of the sprint's build plan and cost model.
  2. Pilot. We build one agent with run IDs, traces, an evaluation set, a review step and a usage and cost dashboard from day one. The pilot is measured against criteria agreed in the sprint.
  3. Production. We add alert tiers, spend and step limits, a kill switch and the incident routine above, with named owners. We deploy where your data lives: Azure AI Foundry, Google Vertex AI, AWS Bedrock or Claude Platform.
  4. Run & Improve. Monitoring, evaluation runs, model and provider updates, prompt tuning, cost control and a monthly report.

We run ByteandRise on a team of agents under the same idea: every agent's work is logged and reviewable, and a person approves each stage. You can read the process in how we run ByteandRise. The AI agent services page describes each stage. To see which of your workflows even needs an agent, try the agent or workflow tool.

Book an AI scoping call. You leave with a written view of what to log and alert on for your workflow, where human review belongs, and a draft incident routine with named owners, so your AI agent observability plan is ready before launch.

FAQ

What is AI agent observability?

AI agent observability is the practice of recording each agent run (inputs, model calls, tool calls, decisions, gate results, cost and human review) so you can explain, replay and fix what the agent did. It goes beyond uptime and latency.

How is it different from LLM observability?

LLM observability covers single model calls: prompts, responses, tokens and latency. Agents add multi-step runs, tool use, handoffs and approvals, so you also need a run-level trace that ties the steps together.

Do we need a special tool, or can we use our existing stack?

Often your existing stack works. OpenTelemetry has generative AI semantic conventions, so traces can flow into a backend you already run. Specialized tools can add evaluation and replay features; the scoping sprint compares options against your data rules.

How do we keep personal data out of agent logs?

Store references and hashes instead of payloads, redact before traces leave the agent process, split access between metadata and raw content, and set retention per field. Check the specifics with your legal team.

Which agent alerts should wake someone up?

Only those where waiting makes things worse: a spend cap hit, an agent acting outside its scope, a tool failing across all runs or a bypassed gate. Trends such as quality score and override rate belong in a weekly review.

Production trailagents with human oversight
step 1seo leadbrief: intent, outline, internal links, sources
step 2writerdraft with a source for every claim
step 3editorfacts, duplicates, brand rules
step 4foundercan override any decision from the admin panel