bad-case-analyzer

Analyze NLP classification predictions to identify FP/FN errors and generate an interactive HTML report.

8|1|Updated Jan 27, 2026
One-click install
npx skills add https://github.com/Difficult-Burger/skills-diy --skill bad-case-analyzer
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: bad-case-analyzer
Source: https://github.com/Difficult-Burger/skills-diy/tree/main/bad-case-analyzer
Command: npx skills add https://github.com/Difficult-Burger/skills-diy --skill bad-case-analyzer

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) and assets (resource) components.

What problem does it solve?

This skill helps data science teams quickly identify and categorize mispredictions (FP/FN) in NLP classification tasks, producing actionable, human-readable HTML reports.

Core Features & Use Cases

  • Metrics: overall accuracy, per-class precision/recall/F1, and confusion matrix
  • Error detection: identify FP/FN and top confusion pairs
  • Interactive HTML reports with filtering by error type, true/predicted label, and confidence
  • Use Case: after evaluating a classifier, generate a report to guide data collection and model improvement

Quick Start

Install the skill, run analysis on a predictions.jsonl file, and open the generated HTML report.

Frequently Asked Questions about bad-case-analyzer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I analyze false positives and false negatives in NLP classification predictions?

To analyze false positives and false negatives in NLP classification, you can process a predictions file to categorize bad cases, compute per-class metrics, and generate an interactive HTML report for error pattern detection.

What do I need to generate a confusion matrix and bad-case report?

You need a predictions file containing fields like text, true_label, and predicted_label, with an optional confidence score. The tool computes the confusion matrix and outputs a bad-case report HTML directly from these inputs.

Can I filter mispredictions by error type and confidence in the evaluation report?

Yes, the generated interactive HTML report supports filtering mispredictions by error type, true or predicted label, and confidence levels, allowing you to isolate specific bad cases for deeper model evaluation and confusion analysis.

How do I identify top confusion pairs after evaluating a classifier?

You can identify top confusion pairs by running a bad-case analysis on your model predictions. The process evaluates the results, builds a confusion matrix, and highlights the most frequent misclassification patterns in the output report.

Does this error analysis tool require specific frameworks or dependencies?

No dependencies are required to use this error analysis tool. It operates independently on a standard predictions file with fields like text, true_label, and predicted_label to compute accuracy and output the interactive HTML report.

What is the best way to visualize model mispredictions for data collection?

The best way to visualize model mispredictions for data collection is to generate an interactive HTML report. It categorizes FP and FN cases, computes per-class F1 scores, and allows filtering to guide targeted model improvement.