What problem does it solve?
This Skill automates independent validation of credit risk machine learning models for banks by producing performance, stability, business, visualization, and regulator-aligned evidence in a single workflow.
Core Features & Use Cases
- End-to-end validation pipeline: covers data quality checks, model scoring, and regulatory readiness for credit risk use cases.
- Model performance metrics: computes core discrimination and classification quality metrics such as AUC, Gini, KS, Recall, Precision, F1, Accuracy, Specificity, and Confusion Matrix.
- Stability analysis (PSI/CSI): evaluates population drift for input features and score drift over time with PSI/CSI thresholds and status labeling.
- Business-oriented cut-off evaluation: estimates approval rate, bad rate, expected loss, and identifies an optimal threshold using business cost logic.
- Audit-ready outputs: generates validation plots (ROC, PR, confusion matrix heatmap, KS, calibration curve, score distributions, PSI charts, lift/gain charts) and compiles them into a final report aligned with Kazakhstan regulatory expectations.
- Optional interactive mode and code review guidance: supports step-by-step validation and a checklist-driven review of model development quality.
Quick Start
Run the full validation flow for a pickled bank credit risk model using your out-of-sample data to generate metrics, PSI/CSI stability outputs, plots, and a regulator-oriented validation report.
