error-analysis

Analyze LLM traces and logs to categorize failures into a structured error taxonomy.

Updated Apr 29, 2026

Search Tags:#analyze #root-cause #failure-analysis #pattern-detection #trace-analysis #llm-traces #error-taxonomy

Ready to run

npx skills add https://github.com/MantisWare/BizForge --skill error-analysis-mantisware

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: error-analysis
Source: https://github.com/MantisWare/BizForge/tree/main/library/skills/analysis/error-analysis
Command: npx skills add https://github.com/MantisWare/BizForge --skill error-analysis-mantisware

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Guides through systematic analysis of LLM execution traces (eval results, agent logs, or production errors) to categorize failure modes, detect recurring patterns, and quantify their frequency, producing an actionable error taxonomy with root-cause hypotheses and prioritized fixes.

Core Features & Use Cases

Ingest traces from eval results, agent logs, and production errors to build a structured view of failures.
Classify failures into standard categories (instruction following, factual error, reasoning error, format violation, refusal, hallucination, context loss, tool misuse) and identify novel patterns.
Quantify patterns across runs to prioritize fixes and guide remediation efforts with actionable hypotheses.

Quick Start

Analyze a set of traces by running /error-analysis with your trace directory to generate an actionable error taxonomy.

Frequently Asked Questions about error-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I analyze LLM traces to find recurring failure patterns?▼

Ingest eval results, agent logs, and production errors to classify failures into standard categories like instruction following, factual error, reasoning error, format violation, refusal, hallucination, context loss, and tool misuse, while identifying novel patterns.

How do I build an error taxonomy from agent logs and evaluation results?▼

Read agent logs and evaluation results to categorize failure modes into standard classes like hallucination, context loss, and tool misuse, quantify pattern frequency, and generate prioritized root-cause hypotheses for remediation.

What is the best way to identify root causes of LLM failures from production logs?▼

Analyze production logs to uncover recurring failure modes, rank them by frequency, and generate prioritized hypotheses for prompt adjustments, guardrails, or data improvements to address LLM failures.

Can I get remediation recommendations for hallucination and tool misuse errors?▼

Analyzing your traces generates prioritized recommendations and hypotheses for remediation, including specific prompt adjustments, guardrails, or data improvements for failures like hallucination and tool misuse.

Does trace analysis work for categorizing reasoning errors and format violations?▼

Trace analysis classifies failures into standard categories including reasoning errors and format violations, detects novel patterns across execution runs, and quantifies their frequency to guide remediation.