skills.rest
DuqueOM
DuqueOMCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

drift-detection

Detect data and concept drift in ML services using PSI and sliced AUC metrics.

5|Updated Nov 18, 2025
Search Tags:#monitoring#drift#data-drift#psi#retraining#auc#concept-drift
One-click install
npx skills add https://github.com/DuqueOM/ML-MLOps-Portfolio --skill drift-detection-duqueom
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: drift-detection
Source: https://github.com/DuqueOM/ML-MLOps-Portfolio/tree/main/.devin/skills/drift-detection
Command: npx skills add https://github.com/DuqueOM/ML-MLOps-Portfolio --skill drift-detection-duqueom

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This drift-detection skill helps data teams detect data drift (PSI) and concept drift (sliced performance) in production ML services, enabling faster diagnosis and informed retraining decisions.

Core Features & Use Cases

  • PSI data-drift detection: monitor feature distribution changes against a reference dataset.
  • Concept-drift diagnosis: analyze sliced performance (AUC, F1) to identify underperforming subgroups.
  • Threshold-driven actions: apply ADR-008 rules to escalate or trigger retraining when drift breaches limits.
  • Operational health: manage drift alerts, heartbeat checks, and collaboration workflows for incident RCA.

Quick Start

Run drift-detection on your production data to compare against the reference, and review PSI and sliced performance results to decide whether retraining is needed.

Frequently Asked Questions about drift-detection

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect data drift in production ML models?▼

You diagnose concept drift by analyzing sliced model performance metrics like AUC and F1. This helps identify underperforming subgroups in production data compared to baseline expectations.

When should I trigger ML model retraining for drift?▼

You should trigger ML model retraining when drift breaches predefined ADR-008 threshold limits. This Skill evaluates PSI and sliced performance metrics to guide automated retraining or escalation decisions.

What is the best way to monitor feature distribution changes?▼

The best way to monitor feature distribution changes is by computing PSI against a reference set. This Skill calculates PSI for features to track distribution shifts and manage operational drift alerts.

Does this drift detection approach work without external dependencies?▼

Yes, this drift detection approach works without external dependencies. It independently computes PSI and sliced AUC metrics to evaluate production data against a reference set.

Why does my model performance drop across different data slices?▼

Model performance drops across data slices due to concept drift. This Skill diagnoses the issue by evaluating sliced AUC metrics against baselines to identify underperforming subgroups.

Related Skills

Explore More Data & Analytics Skills→
mvillmowmvillmow
18

evaluate-model

Evaluate ML model performance on test datasets with classification and regression metrics.

Community
Basic
KentoShimizuKentoShimizu
7

mlops-monitoring-drift

Detect data drift, concept drift, and quality degradation in production ML pipelines.

Community
Advanced
gaofeng21cngaofeng21cn
8

opl-incident-root-cause-triager

Diagnose OPL operational incidents and generate owner-routable root-cause briefs.

Community
Advanced
yonatangrossyonatangross
217

monitoring-observability

Implement Prometheus metrics, Grafana dashboards, and Langfuse tracing for application monitoring.

Community
Advanced