What problem does it solve?
When an LLM agent fails or misbehaves in production (errors, bad answers, latency spikes), finding the root cause across traces, spans, and evaluation signals is slow and unstructured. This Skill provides a step-by-step diagnostic workflow that narrows from errored traces to a verified root cause using the LangWatch CLI.
Core Features & Use Cases
- Errored Trace Discovery: Search production traces with
--errors-only, date windows, text queries, and --jq field extraction to isolate failures and repeated patterns.
- Span-Level Inspection: Read the full span hierarchy of a failing trace to identify which LLM or tool call failed, what prompt the model saw, and whether retries or timeouts are involved.
- Monitor and Evaluator Correlation: Cross-check firing monitors and evaluator scores (toxicity, hallucination, PII) against span evidence, and quantify blast radius with analytics queries.
- Use Case: Users report wrong refund answers from your support agent. You search errored traces, inspect the failing retrieval span, confirm a hallucination monitor is firing, trace the regression to a prompt change via
git log, fix it, and verify errors stop.
Quick Start
Ask the agent to find the root cause of the recent production errors in your LangWatch project and verify the fix with fresh traces.
