datarobot-model-monitoring

Monitor DataRobot deployments for drift, anomalies, and model health.

24|22|Updated Dec 14, 2025
One-click install
npx skills add https://github.com/datarobot-oss/datarobot-agent-skills --skill datarobot-model-monitoring
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: datarobot-model-monitoring
Source: https://github.com/datarobot-oss/datarobot-agent-skills/tree/main/skills/datarobot-model-monitoring
Command: npx skills add https://github.com/datarobot-oss/datarobot-agent-skills --skill datarobot-model-monitoring

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you continuously monitor deployed DataRobot models so you can detect performance degradation, data drift, and prediction anomalies before they impact users.

Core Features & Use Cases

  • Performance monitoring: Track prediction volume, latency, and accuracy-related metrics over time, and compare production metrics to training baselines.
  • Data drift detection: Identify feature drift and target drift (when actuals are available) and quantify drift severity for investigation.
  • Prediction monitoring: Detect unusual prediction patterns and monitor prediction distribution changes and anomaly signals.
  • Model health management: Assess health status, generate monitoring insights, and support alerting and retraining trigger workflows.

Quick Start

Use the skill when you want to check the health of a deployment (for example, deployment abc123) and get a report highlighting any significant drift or anomalies.

Frequently Asked Questions about datarobot-model-monitoring

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor DataRobot deployments for data drift and prediction anomalies?

Data drift detection in DataRobot identifies feature drift and target drift by comparing production distributions to training baselines using the deployment service stats, which helps you quantify drift severity and investigate performance degradation before it impacts users.

How do I set up health alerting for DataRobot MLOps model monitoring?

Model health management in DataRobot generates monitoring insights and supports alerting workflows by assessing deployment health status, evaluating prediction anomalies, and triggering retraining when threshold-based rules detect significant performance degradation or drift.

Can I track prediction latency and accuracy metrics over time for DataRobot models?

Performance monitoring for DataRobot deployments tracks prediction volume, latency, and accuracy metrics over time by comparing production metrics against training baselines, which allows you to validate ongoing prediction quality and detect performance degradation early.

What is the best way to investigate drift events in DataRobot MLOps?

To investigate drift events in DataRobot MLOps, retrieve feature and target drift metrics using the SDK, apply appropriate drift thresholds for severity quantification, and interpret anomaly-focused signals to validate ongoing prediction quality and assess deployment health.

Do I need the DataRobot SDK to monitor deployed model health and detect drift?

Yes, monitoring deployed model health requires the DataRobot SDK to apply MLOps operations like deployment service stats retrieval, feature drift detection, and anomaly interpretation, which are necessary to assess production model performance and trigger alerting workflows.

Why does DataRobot model monitoring show prediction anomalies after deployment?

Prediction anomalies in DataRobot monitoring appear when production prediction distributions change or deviate from training baselines, which signals potential data drift or model health issues that require investigation through anomaly-focused interpretation and drift threshold validation.