skills.rest
yanochka11
yanochka11COMMUNITY
Explore all skills by this author →
GitHub

error-analysis

Identify and categorize failure modes in LLM pipeline traces.

Updated May 5, 2026
Search Tags:#prompt engineering#llm#machine learning#evaluation#error analysis#failure modes#trace analysis
One-click install
npx skills add https://github.com/yanochka11/harness_bro --skill error-analysis-yanochka11
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: error-analysis
Source: https://github.com/yanochka11/harness_bro/tree/main/.claude/skills/curated/evals/error-analysis
Command: npx skills add https://github.com/yanochka11/harness_bro --skill error-analysis-yanochka11

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps teams identify, categorize, and prioritize failure modes in LLM pipelines by systematically analyzing execution traces instead of relying on guesswork.

Core Features & Use Cases

  • Trace Analysis Workflow: Guides users through collecting, reviewing, annotating, and categorizing representative LLM pipeline traces.
  • Failure Categorization: Helps build actionable failure taxonomies, label traces, calculate failure rates, and prioritize improvements.
  • Use Case: Use this Skill after a model change, prompt rewrite, production incident, or evaluation project launch to discover recurring problems and decide what to fix first.

Quick Start

Use the error-analysis skill to analyze my LLM pipeline traces, identify failure categories, and recommend improvement priorities.

Frequently Asked Questions about error-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I identify failure modes in LLM pipeline traces?▼

This process requires collecting representative LLM pipeline traces, reviewing them systematically, annotating failures, and clustering them into actionable categories to prioritize improvements instead of relying on guesswork.

What is the best way to categorize LLM failures after a prompt update?▼

The best way to categorize LLM failures after a prompt update is to build a failure taxonomy, label your execution traces, calculate failure rates, and prioritize iterative system improvements based on the findings.

When do I need to run an error analysis on my machine learning pipeline?▼

You need to run an error analysis on your machine learning pipeline after a model change, prompt rewrite, production incident, or evaluation project launch to discover recurring problems and decide what to fix first.

Can I use trace analysis to fix production incidents in my LLM system?▼

Trace analysis is specifically designed to fix production incidents in LLM systems by reviewing execution logs, categorizing failure modes, and prioritizing iterative system improvements.

Does error analysis work for evaluating prompt engineering changes?▼

Yes, error analysis works for evaluating prompt engineering changes by applying structured trace review and failure labeling to identify recurring problems and prioritize improvements in the LLM pipeline.

Related Skills

Explore More Data & Analytics Skills→
adamatdevopsadamatdevops
1

error-analysis

Analyze LLM pipeline traces to prioritize failure modes by root cause.

Community
Intermediate
AkallabetAkallabet

prompt-engineering-patterns

Apply few-shot learning, chain-of-thought, and structured output patterns to LLM prompts.

Community
Advanced
mcauduro0mcauduro0
1

senior-prompt-engineer

Analyze and optimize LLM prompts with Python scripts.

Community
Advanced
Zack2626-okZack2626-ok

databricks-mlflow-evaluation

Automates MLflow GenAI evaluation with built-in or custom scorers and GEPA prompt optimization.

Community
Advanced