auditing-model-fairness

What problem does it solve?

This Skill audits trained classification and scoring models for fairness disparities across protected groups so you can spot harmful gaps before deployment. It helps teams move beyond a single headline metric and evaluate whether a model treats people differently across race, sex, age, disability, religion, national origin, sexual orientation, or user-defined subgroups.

Core Features & Use Cases

• Per-group fairness analysis: Computes base rates, selection rates, confusion matrices, equal-opportunity gaps, equalized-odds gaps, demographic-parity gaps, calibration checks, and four-fifths-rule ratios.
• Intersectional auditing: Examines combined groups such as race × sex to catch disparities that are hidden in marginal-only reviews.
• Statistical guardrails: Uses bootstrap confidence intervals and low-power warnings for small groups instead of silently dropping them.
• Deployment decision support: Surfaces the trade-offs between fairness criteria and documents which metrics pass or fail without pretending the model is simply fair or unfair.

Quick Start

Use auditing-model-fairness to assess the model with per-group metrics, intersectional cuts, and confidence intervals before deciding whether to deploy it.