experiment-registry-standard

Define and enforce MLflow experiment tracking, registry promotion, and rollback standards.

Updated Jul 16, 2026
One-click install
npx skills add https://github.com/Cloud-Byte-Consulting/plugins --skill experiment-registry-standard-cloud-byte-consulting
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: experiment-registry-standard
Source: https://github.com/Cloud-Byte-Consulting/plugins/tree/main/model-training-ops/skills/experiment-registry-standard
Command: npx skills add https://github.com/Cloud-Byte-Consulting/plugins --skill experiment-registry-standard-cloud-byte-consulting

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Machine learning teams lose reproducibility when runs lack dataset versions, models get copied by hand, and promotions happen without written criteria. This Skill turns MLflow from an installed tool into an enforced organizational standard covering logging contracts, naming, promotion gates, and rollback. ## Core Features & Use Cases - Mandatory logging contract: Defines the fields every run must log (hyperparameters, metrics, dataset name and version, git SHA, environment, seed, model artifact) and rejects incomplete runs at review time. - Registry stage workflow: Specifies the None → Staging → Production → Archived lifecycle with written promotion criteria per transition, CI/CD listeners on stage changes, and archive-never-delete governance. - Rollback runbook: Provides a registry-based rollback procedure that pins deployments by model version instead of latest tags. - Use Case: An ML platform team audits recent models in use, finds runs missing dataset versions, then rolls out a shared MLflow tracking server on Kubernetes with Postgres and S3 artifact storage, a run-linter in CI, and promotion checklists per model family. ## Quick Start Use the experiment-registry-standard skill to audit our current MLflow runs against the logging contract and draft the promotion criteria and rollback runbook for our model registry.

Frequently Asked Questions about experiment-registry-standard

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I enforce MLflow experiment tracking standards across a team?

Define a mandatory logging contract covering hyperparameters, metrics, dataset name and version, git SHA, environment, and seed, then codify it with an init_run helper or a run-linter script that CI runs against new experiments. Reject runs missing any contract field at review time.

What should every MLflow run log for reproducibility?

Every run must log all hyperparameters, objective and secondary metrics, dataset path and version, git SHA and entry point, environment spec, seed, the model artifact via log_model, and evaluation artifacts. The dataset version is the most-skipped and most-regretted field.

How does MLflow model registry promotion and rollback work?

Versions transition through None, Staging, Production, and Archived stages via transition_model_version_stage, with written criteria checked before each move. Rollback transitions the previous good version back to Production and redeploys pinned by version, never by latest tag.

Does MLflow handle data versioning for experiments?

No, MLflow only records the dataset version you tell it; it is not a data version control tool. Pair it with lakeFS, DVC, or immutable versioned prefixes in object storage, and log the dataset URI plus version on every run.

Should I use MLflow autolog or explicit logging?

Prefer explicit logging because autolog coverage is incomplete or experimental for several frameworks in OSS MLflow. Treat autolog as a supplement and verify what it actually captured for your framework version before trusting it.

Why not deploy models using the latest registry version?

Deploying by latest tag means a rollback can race a new registration and cause a second incident. Always pin deployments to a specific model version so rollbacks are deterministic registry operations rather than artifact hunts.