databricks-fsi-predictive-ml

Builds leakage-safe fraud and risk models on Databricks with MLflow tracking and governed scoring.

Updated Aug 28, 2026
One-click install
npx skills add https://github.com/miyake-san/sogo-agent-platform --skill databricks-fsi-predictive-ml-miyake-san
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: databricks-fsi-predictive-ml
Source: https://github.com/miyake-san/sogo-agent-platform/tree/main/skills/experimental/databricks-fsi-predictive-ml
Command: npx skills add https://github.com/miyake-san/sogo-agent-platform --skill databricks-fsi-predictive-ml-miyake-san

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Financial-services ML prototypes often fail because of label leakage, missing baselines, and predictions that downstream layers cannot consume. This Skill guides the full Layer 2 lifecycle of a Databricks FSI solution so a fraud or risk model ends as a versioned, explainable, queryable prediction rather than an untracked notebook. ## Core Features & Use Cases - Leakage-safe feature engineering: Enforces chronological splits, point-in-time rolling features, and training-only preprocessing fits. - FSI evaluation discipline: Requires PR-AUC, calibration, analyst-capacity metrics, slice analysis, and baseline comparisons before model selection. - MLflow operationalization: Logs parameters, metrics, schemas, and thresholds, registers the model, and persists scores with reason codes to a governed prediction table. - Use Case: A solutions engineer building a fraud-detection demo uses this Skill to train a classifier on synthetic transaction data, register it in MLflow, and hand off trace-keyed predictions with reason codes to the GenAI and presentation layers. ## Quick Start Use the databricks-fsi-predictive-ml skill to build a leakage-safe fraud classification model with MLflow tracking and persist scored predictions with reason codes.

Frequently Asked Questions about databricks-fsi-predictive-ml

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prevent data leakage in fraud detection models?

Split data chronologically before fitting any preprocessing, and compute rolling or account-level features only from rows available strictly before the scored event. Fit encoders, imputers, and scalers on training data only, and reject features derived from the target.

What metrics should I use for fraud classification on Databricks?

Report PR-AUC, precision, recall, calibration, and recall or precision at analyst capacity. Compare a naive business baseline and a simple interpretable model against at least one nonlinear candidate on identical splits.

How do I track and register an ML model with MLflow?

Log parameters, metrics, feature schema, training window, code and model artifacts, and the decision threshold to MLflow during training. Then register or otherwise version the selected model so downstream layers can reference it.

Can this approach use protected attributes in credit or risk models?

Protected or sensitive attributes must not be used without documented purpose, review, and controls. The model only recommends or prioritizes; an accountable human or governed policy owns adverse decisions.

What are the limitations of prototype FSI ML models?

Prototype metrics rely on synthetic or historical labels and must be disclosed as such. Expected prevented loss is an explicit hypothesis, not guaranteed ROI, and model probability must never be presented as certainty.