skills.rest
rocklambros
rocklambrosCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

auditing-model-fairness

Audit classification and scoring models for fairness disparities across protected groups.

2|Updated May 23, 2026
Search Tags:#fairness#bias-audit#demographic-parity#bootstrap-ci#intersectionality#equal-opportunity
One-click install
npx skills add https://github.com/rocklambros/rcs --skill auditing-model-fairness
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: auditing-model-fairness
Source: https://github.com/rocklambros/rcs/tree/main/skills/ml-datasci/auditing-model-fairness
Command: npx skills add https://github.com/rocklambros/rcs --skill auditing-model-fairness

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill audits trained classification and scoring models for fairness disparities across protected groups so you can spot harmful gaps before deployment. It helps teams move beyond a single headline metric and evaluate whether a model treats people differently across race, sex, age, disability, religion, national origin, sexual orientation, or user-defined subgroups.

Core Features & Use Cases

  • Per-group fairness analysis: Computes base rates, selection rates, confusion matrices, equal-opportunity gaps, equalized-odds gaps, demographic-parity gaps, calibration checks, and four-fifths-rule ratios.
  • Intersectional auditing: Examines combined groups such as race × sex to catch disparities that are hidden in marginal-only reviews.
  • Statistical guardrails: Uses bootstrap confidence intervals and low-power warnings for small groups instead of silently dropping them.
  • Deployment decision support: Surfaces the trade-offs between fairness criteria and documents which metrics pass or fail without pretending the model is simply fair or unfair.

Quick Start

Use auditing-model-fairness to assess the model with per-group metrics, intersectional cuts, and confidence intervals before deciding whether to deploy it.

Frequently Asked Questions about auditing-model-fairness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit a trained classification model for demographic parity and fairness gaps?▼

To audit for demographic parity and fairness gaps, you need per-record outcomes and protected-attribute vectors. This Skill computes selection rates, confusion matrices, and four-fifths-rule ratios to spot harmful disparities across protected groups before deployment.

Can I check intersectional fairness for combined groups like race and sex?▼

Yes, intersectional auditing examines combined groups like race × sex to catch disparities hidden in marginal-only reviews. It applies intersectional cuts to your protected-attribute vectors to evaluate whether scoring models treat intersectional subgroups differently.

How do I calculate equal opportunity and equalized odds gaps for ML models?▼

You calculate equal opportunity and equalized odds gaps by providing per-record outcomes and protected-attribute labels. The Skill computes base rates and confusion matrices across protected groups to quantify these specific fairness disparities.

What's the best way to handle small sample sizes when measuring model bias?▼

To handle small sample sizes during model bias measurement, the Skill applies bootstrap confidence intervals and issues low-power warnings for small groups. This provides statistical guardrails instead of silently dropping underpowered demographic subgroups.

When do I need to run a four-fifths-rule report for AI model fairness?▼

You need a four-fifths-rule report when evaluating consequential ML decisions in areas like hiring, lending, or healthcare triage. The Skill generates this reporting alongside calibration checks to support deployment decisions for classification models.

Does this fairness audit require calibration checks and protected-attribute labels?▼

Yes, the fairness audit requires protected-attribute labels or user-defined subgroups to run calibration checks. It uses these inputs alongside per-record scores to compute fairness metrics and document which criteria pass or fail.

Related Skills

Explore More Legal & Compliance Skills→
ricablericable

sentence-transformers

Generate normalized text and sentence embeddings for semantic similarity workflows.

Community
Intermediate
xjtulycxjtulyc
33

ethics-ai-analysis

Audit machine learning models for algorithmic fairness violations across sensitive groups.

Community
Advanced
rocklambrosrocklambros
2

evaluating-binary-classifiers

Evaluate binary classifiers with ROC-AUC, PR-AUC, calibration, and bootstrap confidence intervals.

Community
Advanced
HermeticOrmusHermeticOrmus

Model Evaluation Patterns

Generate classification metrics, calibration analysis, and fairness evaluations for machine learning models.

Community
Advanced