agent-observability

To monitor multi-agent system performance and LLM interactions, you need to collect metrics, traces, and logs. This Skill uses OpenTelemetry for tracing agent calls, Prometheus for metrics like latency and token costs, and Grafana for dashboards.

The best way to track LLM token usage and cost is by exporting telemetry data from your agents. This Skill provides token and cost tracking metrics collected via Prometheus, allowing you to monitor usage across various models in your multi-agent system.

To set up alerting for agent call latency and error rates, you configure Alertmanager with Prometheus metrics. This Skill collects performance metrics and integrates with Alertmanager to trigger alerts based on thresholds for concurrent requests and errors.

Yes, you can use OpenTelemetry tracing with Grafana dashboards for agent monitoring. This Skill integrates OpenTelemetry for detailed trace data and Grafana to visualize agent interactions, system health, and performance metrics.

You need OpenTelemetry and Prometheus installed to enable full observability, but structured logging with correlation IDs is generated directly by the Skill. These logs provide traceability for agent activities and LLM calls within the multi-agent system.