agent-observability

Instrument AI agents with trace IDs and child spans for LLM calls.

46|4|Updated Jan 27, 2026

Search Tags:#debugging #monitoring #ai #llm #observability #tracing

Ready to run

npx skills add https://github.com/BagelHole/DevOps-Security-Agent-Skills --skill agent-observability-bagelhole

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: agent-observability
Source: https://github.com/BagelHole/DevOps-Security-Agent-Skills/tree/main/devops/ai/agent-observability
Command: npx skills add https://github.com/BagelHole/DevOps-Security-Agent-Skills --skill agent-observability-bagelhole

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill addresses the challenge of understanding and debugging AI agent performance by providing tools for comprehensive monitoring.

Core Features & Use Cases

Performance Tracking: Monitor request latency, token usage, and tool call success rates.
Cost Management: Track costs per task and per customer.
Reliability & Debugging: Identify issues like hallucinations and retry frequency for improved agent stability.
Use Case: A product team wants to ensure their AI assistant is responsive and cost-effective. They use this Skill to set up alerts for high latency or unexpected token spikes, ensuring a smooth user experience and controlled operational expenses.

Quick Start

Instrument the current AI agent with trace IDs and capture LLM calls as child spans.

Frequently Asked Questions about agent-observability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I monitor AI agent token usage and latency for debugging?▼

You monitor AI agent token usage and latency by instrumenting the agent with trace IDs and capturing LLM calls as child spans. This structured logging tracks request latency, token metrics, and tool call success rates for reliable debugging.

What is AI agent observability and when do I need it?▼

AI agent observability is the process of tracking core signals like request latency, token usage, cost, and hallucination frequency. You need it when troubleshooting agent reliability issues or managing operational expenses for production AI assistants.

How do I track cost per task and customer for my LLM application?▼

You track cost per task and customer by implementing structured tracing that captures token metrics and tool call data. This visibility allows product teams to set alerts for unexpected token spikes and control operational costs.

Can I use this to identify hallucinations and retry frequency in my agent?▼

Yes, you can identify hallucinations and retry frequency by instrumenting your agent with trace IDs and child spans. This reliability monitoring captures tool call failures and retry patterns to improve overall agent stability.

What is the best way to trace tool call success and failure rates in AI agents?▼

The best way to trace tool call success and failure rates is by implementing a tracing pattern with trace IDs and capturing tool calls as child spans. This structured logging provides clear visibility into tool call performance and reliability.