error-analysis

Name: error-analysis
Availability: InStock
Author: kinnerkarmanish

Analyze LLM traces to categorize failures and identify recurring patterns.

Updated Apr 26, 2026

Search Tags:#debugging #taxonomy #error-analysis #pattern-detection #trace-analysis #llm-failures

Ready to run

npx skills add https://github.com/kinnerkarmanish/mak --skill error-analysis-kinnerkarmanish

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: error-analysis
Source: https://github.com/kinnerkarmanish/mak/tree/main/library/skills/analysis/error-analysis
Command: npx skills add https://github.com/kinnerkarmanish/mak --skill error-analysis-kinnerkarmanish

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Guided analysis of LLM traces to categorize failures, detect recurring patterns, and produce an actionable taxonomy that guides improvements in model performance.

Core Features & Use Cases

Ingests eval results, agent logs, and production traces to identify failure modes.
Classifies errors into common categories (reasoning errors, factual errors, format violations, refusals) and surfaces recurring patterns.
Produces root-cause hypotheses and prioritized remediation recommendations to guide engineering and product decisions.

Quick Start

Run error-analysis against a traces folder to generate a taxonomy and actionable remediation recommendations.

Frequently Asked Questions about error-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I analyze LLM traces to categorize failures and detect recurring patterns?▼

Analyze LLM traces by ingesting eval results, agent logs, and production data to classify errors like reasoning failures and format violations, detect recurring patterns, and generate a structured error taxonomy.

What is the best way to diagnose LLM failures from production logs?▼

Diagnose LLM failures from production logs by running a guided trace analysis that classifies errors into categories, surfaces recurring patterns, and produces root-cause hypotheses with prioritized remediation recommendations.

Can I generate an error taxonomy from agent logs and eval results?▼

Yes, you can generate an error taxonomy from agent logs and eval results by ingesting the traces to classify failures into common categories and quantify recurring patterns for actionable improvements.

How do I identify root causes of factual errors and format violations in LLM outputs?▼

Identify root causes of factual errors and format violations by analyzing LLM traces to classify failures, detect recurring patterns, and produce root-cause hypotheses that guide prioritized remediation.

Does this error analysis approach work with agent logs and production traces?▼

Yes, this error analysis approach works with agent logs and production traces by ingesting them to identify failure modes, classify errors, and generate actionable remediation recommendations.

What types of LLM failures can be classified using trace analysis?▼

Trace analysis classifies LLM failures into common categories including reasoning errors, factual errors, format violations, and refusals, while surfacing recurring patterns to produce a structured error taxonomy.