skills.rest
ehadziabdic
ehadziabdicCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

ml-experiment-tracking

Track and compare ML experiments with MLflow, Weights & Biases, Neptune, and CometML.

Updated Sep 23, 2026
Search Tags:#machine learning#hyperparameter tuning#reproducibility#experiment tracking#mlflow#model registry#weights and biases
One-click install
npx skills add https://github.com/ehadziabdic/WAgents --skill ml-experiment-tracking-ehadziabdic
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ml-experiment-tracking
Source: https://github.com/ehadziabdic/WAgents/tree/main/opencode/skills/ml-experiment-tracking
Command: npx skills add https://github.com/ehadziabdic/WAgents --skill ml-experiment-tracking-ehadziabdic

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires mlflow, matplotlib, numpy, scikit-learn, and includes scripts (resource) and references (resource) components.

What problem does it solve? Machine learning experiments produce scattered hyperparameters, metrics, and artifacts that are hard to reproduce, compare, or share across a team. This Skill provides structured logging, comparison, and reproducibility workflows so every training run is recorded and queryable. ## Core Features & Use Cases - Experiment Logging: Log hyperparameters, metrics, artifacts, and models with MLflow or Weights & Biases, including auto-logging for PyTorch, TensorFlow, scikit-learn, XGBoost, and LightGBM. - Comparison & Analysis: Query, rank, and statistically compare runs, generate markdown/JSON reports, and produce parallel-coordinates plots for hyperparameter searches. - Reproducibility & Infrastructure: Capture git commits, environment files, data fingerprints, and seeds, and set up remote tracking servers with PostgreSQL and S3 or managed MLflow on SageMaker. - Use Case: While tuning a churn model with Optuna, log each trial as a nested run, then run the comparison script to identify the best configuration and promote it to the model registry. ## Quick Start Ask the AI to set up MLflow tracking for your training script with auto-logging enabled and a comparison report of the top runs by accuracy.

Frequently Asked Questions about ml-experiment-tracking

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I track ML experiments with MLflow in Python?▼

Set a tracking URI, create an experiment with mlflow.set_experiment, then wrap training in mlflow.start_run to log parameters, metrics, artifacts, and models. Enable framework auto-logging such as mlflow.sklearn.autolog() to capture runs without manual logging calls.

MLflow vs Weights & Biases for experiment tracking?▼

MLflow is open source and self-hosted with a built-in model registry, suiting full-lifecycle tracking. Weights & Biases is a freemium SaaS with richer visualizations, sweep UIs, and team reports, better for collaborative research.

How to compare MLflow experiment runs by metric?▼

Use MlflowClient.search_runs with a filter string and order_by clause, or run the experiment_compare.py script with --experiment and --metric flags. It outputs ranked tables, markdown or JSON reports, and hyperparameter correlation statistics.

Does MLflow auto-logging work with HuggingFace Transformers?▼

There is no mlflow.transformers.autolog() function. Use the generic mlflow.autolog() or attach the transformers MLflowCallback to the Trainer to capture training metrics and parameters automatically.

How do I make ML experiments reproducible?▼

Log the git commit hash, environment files like requirements.txt or conda.yaml, a hash of the training data, and all random seeds as run parameters or tags. This lets any run be recreated exactly from its recorded metadata.

Can I run an MLflow tracking server for my team?▼

Yes, start mlflow server with a PostgreSQL backend-store-uri and an S3 default-artifact-root for production-grade shared tracking. On AWS, managed MLflow via SageMaker provides the same setup without operating the database yourself.

Related Skills

Explore More Data & Analytics Skills→
organvm-i-theoriaorganvm-i-theoria

ml-experiment-tracker

Track ML experiments, version datasets, and manage model registries with MLflow, W&B, and DVC.

Official
Intermediate
qcmuuqcmuu

mlflow

Track ML experiments, version models, and deploy via MLflow registry URIs.

Community
Advanced
dominodatalabdominodatalab
6

domino-experiment-tracking

Track MLflow experiments in Domino Data Lab with auto-logging.

Official
Intermediate
mlflowmlflow
69

sagemaker-mlflow

Connect SageMaker environments to MLflow for tracking experiments and managing models.

Official
Advanced