skills.rest
MantisWare
MantisWareCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

error-analysis

Analyze LLM traces and logs to categorize failures into a structured error taxonomy.

Updated Apr 29, 2026
Search Tags:#analyze#root-cause#failure-analysis#pattern-detection#trace-analysis#llm-traces#error-taxonomy
One-click install
npx skills add https://github.com/MantisWare/BizForge --skill error-analysis-mantisware
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: error-analysis
Source: https://github.com/MantisWare/BizForge/tree/main/library/skills/analysis/error-analysis
Command: npx skills add https://github.com/MantisWare/BizForge --skill error-analysis-mantisware

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Guides through systematic analysis of LLM execution traces (eval results, agent logs, or production errors) to categorize failure modes, detect recurring patterns, and quantify their frequency, producing an actionable error taxonomy with root-cause hypotheses and prioritized fixes.

Core Features & Use Cases

  • Ingest traces from eval results, agent logs, and production errors to build a structured view of failures.
  • Classify failures into standard categories (instruction following, factual error, reasoning error, format violation, refusal, hallucination, context loss, tool misuse) and identify novel patterns.
  • Quantify patterns across runs to prioritize fixes and guide remediation efforts with actionable hypotheses.

Quick Start

Analyze a set of traces by running /error-analysis with your trace directory to generate an actionable error taxonomy.

Frequently Asked Questions about error-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I analyze LLM traces to find recurring failure patterns?▼

Ingest eval results, agent logs, and production errors to classify failures into standard categories like instruction following, factual error, reasoning error, format violation, refusal, hallucination, context loss, and tool misuse, while identifying novel patterns.

How do I build an error taxonomy from agent logs and evaluation results?▼

Read agent logs and evaluation results to categorize failure modes into standard classes like hallucination, context loss, and tool misuse, quantify pattern frequency, and generate prioritized root-cause hypotheses for remediation.

What is the best way to identify root causes of LLM failures from production logs?▼

Analyze production logs to uncover recurring failure modes, rank them by frequency, and generate prioritized hypotheses for prompt adjustments, guardrails, or data improvements to address LLM failures.

Can I get remediation recommendations for hallucination and tool misuse errors?▼

Analyzing your traces generates prioritized recommendations and hypotheses for remediation, including specific prompt adjustments, guardrails, or data improvements for failures like hallucination and tool misuse.

Does trace analysis work for categorizing reasoning errors and format violations?▼

Trace analysis classifies failures into standard categories including reasoning errors and format violations, detects novel patterns across execution runs, and quantifies their frequency to guide remediation.

Related Skills

Explore More Data & Analytics Skills→
Miosa-osaMiosa-osa
226

error-analysis

Analyze LLM execution traces to categorize failures and generate an error taxonomy.

Official
Advanced
HanHan666666HanHan666666
10

systematic-debugging

Investigate root causes of bugs through a structured four-phase debugging process.

Community
Advanced
DhumitechDhumitech

agent-evaluation

Automate AI agent evaluation and benchmarking with repeatable test pipelines.

Community
Advanced
LincyawLincyaw

langgraph-trajectory

Record, export, and replay LangGraph execution trajectories with checkpoint history.

Community
Advanced