evaluating-binary-classifiers

What problem does it solve?

This Skill turns a trained binary classifier into a defensible evaluation report so you can judge real performance instead of relying on a single misleading number.

Core Features & Use Cases

• Balanced and imbalanced evaluation: Checks class balance first and reframes the report around PR-AUC when the positive class is rare.
• Threshold-aware analysis: Compares multiple thresholds, avoids defaulting to 0.5, and selects an operating point using cost-aware or precision-floor rules.
• Calibration and uncertainty: Adds reliability analysis, Brier score, confusion matrices, and bootstrap confidence intervals for key metrics.
• Use case: Review a spam, fraud, or medical screening model from held-out predictions and produce a complete, production-style evaluation.

Quick Start

Ask for a full binary-classifier evaluation from y_true and y_pred_proba, including class balance, ROC and PR analysis, calibration, threshold selection, and bootstrap confidence intervals.