instrument-llm-observability

To add tracing to an LLM application, you need production-grade instrumentation that captures prompts, tool calls, token usage, latency, and cost per step, ensuring every run is fully inspectable for monitoring.

OpenTelemetry works for LLM observability by providing vendor-neutral GenAI semantic conventions, enabling tracing integration that captures cost per span and latency data across various LLM SDKs in AI agent applications.

To track cost per step when debugging an LLM app in production, implement observability instrumentation that captures cost per span and latency data, allowing you to inspect detailed performance metrics for each run.

You can create custom spans for tool and retrieval steps in an AI agent through custom instrumentation, allowing manual span creation to capture specific operations and enhance tracing observability.

The best way to instrument an AI agent for cost and performance tracking is using vendor-neutral OpenTelemetry tracing, which captures cost per span, latency data, and tool calls to ensure every run is fully inspectable.