ml-rigor

Automate ML evaluation with baselines, stratified cross-validation, and leakage checks.

242|64|Updated Jan 2, 2026
One-click install
npx skills add https://github.com/Yeachan-Heo/My-Jogyo --skill ml-rigor
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ml-rigor
Source: https://github.com/Yeachan-Heo/My-Jogyo/tree/main/skills/ml-rigor
Command: npx skills add https://github.com/Yeachan-Heo/My-Jogyo --skill ml-rigor

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Enforces baseline comparisons, cross-validation, interpretation, and leakage prevention for ML pipelines.

Core Features & Use Cases

  • Baseline-driven evaluation: require comparison to a dummy or simple model before claiming progress.
  • Comprehensive cross-validation: report mean and std across stratified folds, with confidence intervals.
  • Model interpretation: provide permutation importance and SHAP analyses to explain predictions.
  • Leakage prevention & error analysis: detect data leakage, slice performance by segments, and analyze failure modes.

Quick Start

Load your dataset, define a model, and run the ml-rigor workflow to generate baselines, CV metrics, and interpretation artifacts ready for reporting.

Frequently Asked Questions about ml-rigor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prevent data leakage in my ML evaluation pipeline?

Cross-validation prevents overfitting by splitting data into multiple stratified folds, training on some, and validating on others. This skill automates stratified CV, reporting mean and std metrics with confidence intervals for reliable baseline comparisons.

Why do I need a baseline model comparison before claiming ML progress?

Baseline model comparisons are required to prove a new model adds value over a simple dummy or heuristic model. This skill enforces baseline-driven evaluation, generating comparison reports before you claim any performance improvements.

What's the best way to interpret model predictions using permutation importance?

Permutation importance interprets predictions by measuring the drop in model score when a feature's values are shuffled. This skill provides permutation importance and SHAP analyses to generate interpretable reports explaining feature impact.

How do I run stratified cross-validation for classification and regression tasks?

You can run stratified cross-validation for classification and regression by defining your model and dataset in the workflow. The skill automates CV across small to large datasets, outputting fold summaries and error analysis artifacts.

Does this ML model evaluation workflow support slicing performance by data segments?

Yes, ML model evaluation supports slicing performance by data segments to analyze failure modes. The skill's error analysis capabilities detect leakage and break down metrics across different data segments within a reproducible workflow.