agent-observability

Name: agent-observability
Availability: InStock
Author: ils15

Monitor multi-agent system performance with OpenTelemetry, Prometheus, and Alertmanager.

10|5|Updated Dec 12, 2025

Search Tags:#Prometheus #Grafana #agent monitoring #OpenTelemetry #Alertmanager #LLM performance

Ready to run

npx skills add https://github.com/ils15/pantheon --skill agent-observability-ils15

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: agent-observability
Source: https://github.com/ils15/pantheon/tree/main/platform/windsurf/.windsurf/skills/agent-observability
Command: npx skills add https://github.com/ils15/pantheon --skill agent-observability-ils15

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires opentelemetry, prometheus, grafana, alertmanager, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill ensures observability and performance tracking across multi-agent systems, aiding in monitoring agent interactions, LLM calls, and system health.

Core Features & Use Cases

OpenTelemetry Tracing: Provides detailed trace data for agent and LLM interactions.
Token & Cost Tracking: Monitors and tracks token usage and costs for various models.
Agent Performance Metrics: Collects and presents metrics on agent call latency, error rates, and concurrent requests.
Structured Logging: Generates structured logs with correlation IDs for traceability.
Alerting & Dashboards: Integrates with Alertmanager and Grafana for alerting and monitoring dashboards.

Quick Start

Enable the agent-observability skill within your multi-agent system to monitor and log agent activities.

Frequently Asked Questions about agent-observability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I monitor multi-agent system performance and LLM interactions?▼

To monitor multi-agent system performance and LLM interactions, you need to collect metrics, traces, and logs. This Skill uses OpenTelemetry for tracing agent calls, Prometheus for metrics like latency and token costs, and Grafana for dashboards.

What is the best way to track LLM token usage and cost across agents?▼

The best way to track LLM token usage and cost is by exporting telemetry data from your agents. This Skill provides token and cost tracking metrics collected via Prometheus, allowing you to monitor usage across various models in your multi-agent system.

How do I set up alerting for agent call latency and error rates?▼

To set up alerting for agent call latency and error rates, you configure Alertmanager with Prometheus metrics. This Skill collects performance metrics and integrates with Alertmanager to trigger alerts based on thresholds for concurrent requests and errors.

Can I use OpenTelemetry tracing with Grafana dashboards for agent monitoring?▼

Yes, you can use OpenTelemetry tracing with Grafana dashboards for agent monitoring. This Skill integrates OpenTelemetry for detailed trace data and Grafana to visualize agent interactions, system health, and performance metrics.

Do I need OpenTelemetry and Prometheus to enable structured logging for agents?▼

You need OpenTelemetry and Prometheus installed to enable full observability, but structured logging with correlation IDs is generated directly by the Skill. These logs provide traceability for agent activities and LLM calls within the multi-agent system.