evaluating-ood-detection

What problem does it solve?

This Skill evaluates whether a classifier can recognize inputs that do not belong to its training distribution, so teams can add a reliable rejection layer before deployment.

Core Features & Use Cases

• Open-world OOD evaluation: Separates in-distribution, near-OOD, and far-OOD inputs so the hardest deployment case is measured honestly.
• Method selection and comparison: Guides you through energy score, Mahalanobis distance, KNN distance, ODIN, and max-softmax baselines without defaulting to the weakest option.
• Risk-aware reporting: Produces AUROC, AUPR-out, FPR at 95 percent TPR, bootstrap confidence intervals, and a deployment threshold tied to false-accept versus false-reject cost.
• Use case: If you are shipping an image classifier that may see novel classes in production, this Skill helps you test the detector on held-out in-domain data plus near-OOD and far-OOD sets and report the results separately.

Quick Start

Ask this skill to evaluate your classifier on held-out in-distribution data, at least one near-OOD set, and any far-OOD sets you have, then choose an OOD scoring method and a deployment threshold.