failure-taxonomy

What problem does it solve?

This Skill transforms unstructured, freeform annotations from LLM trace reviews into a structured, actionable taxonomy of failure modes, enabling systematic error analysis and improvement.

Core Features & Use Cases

• Axial Coding: Groups open-coded annotations into coherent, non-overlapping binary failure categories.
• Taxonomy Building: Defines clear titles, definitions, and examples for each failure mode.
• Re-labeling & Quantification: Applies the taxonomy to traces and calculates error rates for prioritization.
• Use Case: After reviewing 50 user interactions with a chatbot, you have raw notes like "bot misunderstood intent" or "gave irrelevant info". This Skill helps you cluster these into categories like "Intent Misinterpretation" or "Off-Topic Response" and quantify how often each occurs.

Quick Start

Use the failure-taxonomy skill to build a taxonomy from the provided annotations.