skills.rest
RHEcosystemAppEng
RHEcosystemAppEngOFFICIAL
Explore all skills by this author →
GitHub

model-monitor

Monitor deployed InferenceServices for bias and data drift using SPD, DIR, and statistical metrics.

48|31|Updated Feb 2, 2026
Search Tags:#bias detection#data drift#model monitoring#TrustyAI#InferenceService
One-click install
npx skills add https://github.com/RHEcosystemAppEng/agentic-plugins --skill model-monitor-rhecosystemappeng
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-monitor
Source: https://github.com/RHEcosystemAppEng/agentic-plugins/tree/main/rh-ai-engineer/skills/model-monitor
Command: npx skills add https://github.com/RHEcosystemAppEng/agentic-plugins --skill model-monitor-rhecosystemappeng

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires resources_get, resources_list, resources_create_or_update, pods_list, pods_log, events_list, list_inference_services, get_inference_service, list_data_science_projects, execute_promql, and includes scripts (resource) and references (resource) components.

What problem does it solve?

Detects bias and data drift in deployed models, ensuring model fairness and performance integrity.

Core Features & Use Cases

  • Bias Detection: Monitor statistical parity differences (SPD) and disparate impact ratios (DIR) to identify biased outcomes.
  • Drift Detection: Use metrics like Mean Shift, Fourier MMD, KS-Test, and Jensen-Shannon to detect changes over time.
  • Use Case: Ensure your model maintains fairness and performance post-deployment by setting up model monitoring on InferenceServices.

Quick Start

Monitor bias and drift for the 'inference-service-1' model by running the 'model-monitor' skill.

Frequently Asked Questions about model-monitor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect bias and data drift in deployed InferenceServices?▼

You can detect bias and data drift in deployed InferenceServices by monitoring statistical parity differences (SPD) and disparate impact ratios (DIR) for bias, alongside Mean Shift, Fourier MMD, KS-Test, and Jensen-Shannon metrics for drift detection.

What metrics are used for model fairness and data drift monitoring?▼

Model fairness monitoring uses statistical parity difference (SPD) and disparate impact ratio (DIR) to identify biased outcomes, while data drift detection uses Mean Shift, Fourier MMD, KS-Test, and Jensen-Shannon metrics to track distributional changes over time.

Do I need a TrustyAIService to monitor model fairness on OpenShift?▼

Yes, monitoring model fairness and data drift requires access to the TrustyAIService CRD within an OpenShift cluster, utilizing RHOAI tools for necessary data collection and continuous monitoring of your deployed models.

How do I set up drift detection for a machine learning model post-deployment?▼

To set up drift detection post-deployment, you configure monitoring on your deployed InferenceService to continuously evaluate data using Mean Shift, Fourier MMD, KS-Test, and Jensen-Shannon metrics to detect performance changes over time.

Can I use PromQL to monitor inference service performance and drift?▼

Yes, the monitoring process supports executing PromQL queries to collect metrics and evaluate data drift and bias detection for InferenceServices running within your OpenShift data science projects.

Why does model monitoring fail to collect fairness metrics on my cluster?▼

Model monitoring fails if the required TrustyAIService CRD is not properly configured on your OpenShift cluster, preventing the RHOAI tools from collecting the necessary bias and drift data from your deployed InferenceServices.

Related Skills

Explore More Data & Analytics Skills→
RHEcosystemAppEngRHEcosystemAppEng
48

ds-project-setup

Automate Data Science Project setup on OpenShift AI with namespaces, S3 connections, pipelines, and model serving.

Official
Advanced
nirukk52nirukk52
1

dev-log-monitoring

Monitor Encore.ts and SvelteKit logs to diagnose drift and verify run events.

Community
Advanced
RHEcosystemAppEngRHEcosystemAppEng
48

debug-pod

Diagnose OpenShift pod failures by collecting status, events, logs, and resource information.

Official
Intermediate
HermeticOrmusHermeticOrmus

Model Evaluation Patterns

Generate classification metrics, calibration analysis, and fairness evaluations for machine learning models.

Community
Advanced