rollback

Revert production ML deployments during incidents using Argo Rollouts or Kubernetes.

5|Updated Nov 18, 2025
One-click install
npx skills add https://github.com/DuqueOM/ML-MLOps-Portfolio --skill rollback-duqueom
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: rollback
Source: https://github.com/DuqueOM/ML-MLOps-Portfolio/tree/main/.devin/skills/rollback
Command: npx skills add https://github.com/DuqueOM/ML-MLOps-Portfolio --skill rollback-duqueom

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Emergency rollback of production ML deployments to a known-good state during incidents, preventing user impact and data loss by aborting in-flight changes and reverting deployments and models.

Core Features & Use Cases

  • Abort in-flight changes: stops ongoing canary or rollout processes to halt bad releases.
  • Targeted recovery: reverts to the last healthy revision and, if needed, rolls back model artifacts in MLflow.
  • Audit-ready process: requires human approval for destructive steps and records an incident audit trail.

Quick Start

Initiate an emergency rollback by identifying the affected service, selecting the last healthy revision, executing the rollback plan (Argo Rollouts or kubectl), reverting the MLflow model if needed, silencing alerts, and validating pod readiness

Frequently Asked Questions about rollback

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I rollback a production ML deployment during an active incident?

To rollback a production ML deployment during an incident, you need an approved plan to abort canaries, undo the deployment, revert the MLflow model stage, silence alerts, and audit the rollback. This restores service stability by reverting to the last healthy revision.

What is the process to revert an MLflow model stage during a Kubernetes incident?

Reverting an MLflow model stage during a Kubernetes incident involves applying a STOP-approved plan that aborts in-flight changes and rolls back model artifacts. This targeted recovery ensures your production ML service returns to a known-good state without data loss.

Does emergency rollback work with both Argo Rollouts and plain Kubernetes Deployments?

Yes, emergency rollback works with both Argo Rollouts and plain Kubernetes Deployments. It executes the rollback plan using the respective controller mechanisms to revert to the last healthy revision and halt bad releases effectively.

How do I abort in-flight canary releases to stop a bad ML model deployment?

To abort in-flight canary releases and stop a bad ML model deployment, execute an emergency rollback plan that halts ongoing rollout processes. This action immediately stops bad releases and prevents further user impact across your production environment.

What steps are required to validate pod readiness after a production ML rollback?

Validating pod readiness after a production ML rollback is the final step in the recovery process. Once you revert the deployment and MLflow model stage, you must verify that the pods stabilize to ensure the service is fully restored.

When should I silence alerts during an incident rollback?

You should silence alerts during an incident rollback after undoing the deployment and reverting the MLflow model stage. This prevents alert fatigue while you validate pod readiness and complete the audit trail for the emergency rollback.