domino-model-monitoring

Model monitoring captures predictions from your deployed Model API and analyzes them against a training dataset baseline to detect data drift and performance degradation. It optionally compares predictions against ground truth data to assess model quality metrics.

Concept drift occurs when the relationship between input data and target predictions changes in production. Model monitoring identifies this drift and configures alerts to notify teams when quality metrics breach predefined thresholds, enabling proactive model management.

To set up monitoring for a deployed Model API in Domino Data Lab, you need a deployed Model API and a training dataset for baseline comparison. The monitoring skill captures predictions, analyzes drift, and tracks quality metrics against optional ground truth data.

Ground truth data is optional for monitoring model performance and prediction drift. You can detect data drift and capture predictions using only a deployed Model API and a training dataset baseline, but ground truth is required to assess model quality metrics.

The best way to track prediction accuracy drops is by logging model predictions for analysis and comparing them against ground truth data. This model monitoring approach automatically detects significant accuracy drops and alerts the MLOps team to initiate retraining.

Concept drift detection requires a training dataset baseline to compare against incoming production data distributions. Without this baseline, the monitoring cannot identify changes in input data distributions or accurately assess prediction drift for alert configuration.