What problem does it solve?
When analyzing LLM traces, failure patterns are unknown before a taxonomy exists, making it hard to label spans consistently. This Skill provides a qualitative coding workflow: inspect spans, write evidence-close open notes, recover them, group recurring patterns, and only then create structured annotations.
Core Features & Use Cases
- Open Coding with PXI Notes: Write durable, per-span observations using the server-executed
write_span_note tool with the pxi identifier, keeping notes as the canonical source of truth.
- Note Recovery via GraphQL: Paginate
Project.spans with the phoenix-graphql skill and filter spanNotes to identifier == "pxi" to resume analysis across sessions.
- Axial Coding and Annotation Promotion: Group recovered notes into named categories, check existing annotation configs, and apply
ui.spans.annotate only after labels stabilize.
- Use Case: An engineer investigating why a support chatbot fails samples 200 spans, writes open notes like "Retriever returned onboarding docs for a cancellation question," groups them into a retrieval-mismatch category, and then applies a structured annotation label across the project.
Quick Start
Ask the agent to open-code a sample of spans from your Phoenix project to discover failure patterns before defining any annotation taxonomy.
