experiment-compare

Compare MLflow experiment runs across metrics, hyperparameters, tags, and artifacts.

Updated Mar 20, 2026
One-click install
npx skills add https://github.com/thbraet/claude-template --skill experiment-compare
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: experiment-compare
Source: https://github.com/thbraet/claude-template/tree/main/skills/experiment-compare
Command: npx skills add https://github.com/thbraet/claude-template --skill experiment-compare

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Load MLflow experiment runs and produce a structured, side-by-side comparison across metrics, hyperparameters, feature usage, and training characteristics to support model selection.

Core Features & Use Cases

  • Compare metrics (accuracy, F1, AUC, RMSE) across runs.
  • Compare hyperparameters and feature sets to identify drivers of performance.
  • Generate a reproducible comparison notebook and a concise recommendation for stakeholders.

Quick Start

Provide a set of run IDs, run names, or 'latest N' to load and compare MLflow runs.

Frequently Asked Questions about experiment-compare

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare MLflow runs to select the best model?

You can compare MLflow experiment runs by providing specific run IDs, run names, or selecting the latest N runs. The tool aggregates metrics, hyperparameters, and artifacts into a structured side-by-side report to justify the best model choice.

What metrics can I compare across MLflow experiments?

You can compare metrics such as accuracy, F1, AUC, and RMSE across MLflow experiment runs. The comparison also aggregates hyperparameters, feature sets, and training characteristics to identify what drives model performance.

Can I include feature importance files and confusion matrices when comparing MLflow runs?

Yes, you can include optional artifacts like feature importance files or confusion matrices when evaluating MLflow runs. These artifacts are aggregated alongside metrics and hyperparameters to provide a comprehensive model evaluation.

Do I need a specific environment to compare MLflow runs for model selection?

You need access to an MLflow tracking server with existing experiment runs that have logged metrics and parameters. Optional artifacts like feature importance files or confusion matrices can be included if available.

What is the best way to document MLflow model selection decisions for stakeholders?

The best way to document MLflow model selection decisions is to generate a reproducible comparison notebook alongside a concise recommendation. This aggregates run metrics and hyperparameters into a structured report for stakeholders.