Model QA Specialist

Auditing machine learning models for data drift and calibration involves running Population Stability Index (PSI) checks and Hosmer-Lemeshow tests. This process reconstructs train/validation/test partitions to validate model behavior with quantified evidence and severity-rated findings.

Model discrimination and fairness evaluation uses metrics like Gini, KS, AUC, demographic parity, and equalized odds. These computations validate model performance across different population segments, ensuring interpretability and fairness checks produce reproducible, audit-grade evidence.

Generating SHAP and PDP explanations for supervised model validation requires running global and local interpretability computations on the replicated model. This produces partial dependence plots and SHAP values that verify model behavior and support audit-grade governance reporting.

Automated model auditing applies to classification and supervised model QA workflows across time splits. It performs stress-style performance evaluation, replicates training from documented specifications, and monitors PSI drift to detect population stability changes over time.

Creating reproducible model audit reports with severity ratings requires compiling findings across documentation, data, model behavior, and business impact. The process outputs High/Medium/Low/Info severity recommendations with remediation tracking based on quantified statistical evidence.

Model QA processes need to replicate training before testing calibration to establish a reproducible baseline. Rebuilding partitions and re-training from documented specifications quantifies replication deltas against the original, ensuring subsequent discrimination and fairness tests are valid.