cortex-eval

Evaluate model performance, drift, and error patterns on production data.

69|8|Updated Mar 16, 2026

Search Tags:#production #metrics #drift #model #error-analysis #evaluate #ml-ops

Ready to run

npx skills add https://github.com/tonone-ai/tonone --skill cortex-eval

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: cortex-eval
Source: https://github.com/tonone-ai/tonone/tree/main/team/cortex/skills/cortex-eval
Command: npx skills add https://github.com/tonone-ai/tonone --skill cortex-eval

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Evaluate model performance, drift, and error patterns on production data to identify degradations and strategies for remediation.

Core Features & Use Cases

Performance Evaluation: compare current metrics against baselines to detect degradation.
Drift & Error Analysis: analyze data drift, feature shifts, and failure modes across cohorts.
Use Case: monitor a deployed classifier after retraining to confirm improvements or identify regression.

Quick Start

Evaluate the latest production data against the current model to surface actionable insights.

Frequently Asked Questions about cortex-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I evaluate model performance and detect drift on production ML data?▼

To evaluate model performance and detect drift, you compare current production metrics against established baselines. This process analyzes data shifts and error patterns across segments to identify model degradation and inform remediation strategies.

What is the best way to monitor a deployed classifier for regression after retraining?▼

Monitoring a deployed classifier for regression requires comparing post-training metrics against previous baselines. By analyzing prediction drift and error patterns on production data, you can confirm if retraining improved performance or introduced new failures.

How does error analysis on production data help debug ML model degradation?▼

Error analysis on production data helps debug ML model degradation by isolating failure modes across specific data cohorts. It reveals underlying feature shifts and environment changes causing performance drops, enabling targeted remediation.

Can I validate model behavior across different data segments in production?▼

You can validate model behavior across different data segments by applying an evaluation workflow to production data. This analyzes performance variations and prediction drift across cohorts to ensure consistent system reliability.

What metrics are needed to compare baseline vs current performance for ML monitoring?▼

Comparing baseline vs current performance for ML monitoring requires production metrics that capture prediction accuracy and data distributions. Analyzing these metrics highlights data drift and error patterns, generating a structured report of system health.

Why does my deployed ML model show performance degradation despite successful training?▼

Deployed ML model performance degradation often stems from data drift or environment changes in production. Comparing current metrics against baselines and analyzing feature shifts across cohorts identifies the specific failure modes causing the regression.