skills.rest
pluginagentmarketplace
pluginagentmarketplaceCOMMUNITY
Explore all skills by this author →
GitHub

ml-monitoring

Detect data and model drift in production ML deployments.

1|Updated Nov 18, 2025
Search Tags:#monitoring#observability#alerts#drift#ML#A/B-testing#data-drift
One-click install
npx skills add https://github.com/pluginagentmarketplace/custom-plugin-mlops --skill ml-monitoring
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ml-monitoring
Source: https://github.com/pluginagentmarketplace/custom-plugin-mlops/tree/main/skills/ml-monitoring
Command: npx skills add https://github.com/pluginagentmarketplace/custom-plugin-mlops --skill ml-monitoring

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires PyYAML, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

Production-grade ML monitoring addresses the lack of visibility into model performance, data drift, and observability in live deployments.

Core Features & Use Cases

  • Drift detection, data quality monitoring, and alerting to prevent degraded models
  • Observability dashboards and reports to support rapid root cause analysis
  • Real-world scenario: detect data drift between training and production data and trigger alerts with remediation suggestions

Quick Start

Invoke the ml-monitoring skill to initialize drift detection on a deployed model and generate a drift report.

Frequently Asked Questions about ml-monitoring

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect data drift in a production ML model?▼

Data drift detection compares training and production data to identify distribution shifts. You configure drift_config and thresholds to generate drift reports, trigger alerts, and receive remediation suggestions for degraded models.

What is ML observability and how does it prevent model degradation?▼

ML observability provides visibility into live model performance, data drift, and quality gaps. By monitoring batch and streaming inference continuously, it enables rapid root cause analysis and prevents degraded models.

Can I set up alerts for model drift in streaming inference pipelines?▼

Yes, model drift alerting applies across both batch and streaming inference. You define monitoring_type and thresholds to automatically trigger alerts with remediation suggestions when drift is detected in live deployments.

How do I generate a drift report for a deployed machine learning model?▼

Invoke the monitoring skill on your deployed model with structured input for monitoring_type, drift_config, and thresholds. The output includes detailed drift reports, alerts, and actionable recommendations for remediation.

Does A/B testing work with ML monitoring for experimentation inputs?▼

Yes, ML monitoring supports A/B testing by providing experimentation inputs. By detecting data and model drift across batch and streaming inference, it supplies observability dashboards and reports that feed directly into experimentation workflows.

Related Skills

Explore More Data & Analytics Skills→
ToXMonToXMon

domino-model-monitoring

Detect data drift and monitor model health for Domino-deployed models.

Community
Advanced
MuzeWinterMuzeWinter
1

analyze-data-quality

Assesses dataset trustworthiness for analysis, modeling, dashboards, experiments, or downstream pipelines.

Community
Advanced
microwindmicrowind
65

data-pipeline

Ingest, transform, and load data from multiple sources into warehouses.

Official
Advanced
ContextJet-aiContextJet-ai
29

set-up-drift-alerts

Set up drift alerts for LLM apps using OpenTelemetry GenAI semantic conventions.

Official
Intermediate