skills.rest
archibate
archibateCOMMUNITY
Explore all skills by this author →
Website𝕏@archibateGitHub

ml-pipeline-workflow

Build end-to-end MLOps pipelines automating ingestion, training, validation, deployment, and monitoring.

6|Updated Feb 25, 2026
Search Tags:#deployment#mlops#orchestration#pipeline#training#experiment-tracking#model-serving
One-click install
npx skills add https://github.com/archibate/archibate-skills --skill ml-pipeline-workflow-archibate
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ml-pipeline-workflow
Source: https://github.com/archibate/archibate-skills/tree/main/old-skills/trash-skills/ml-pipeline-workflow
Command: npx skills add https://github.com/archibate/archibate-skills --skill ml-pipeline-workflow-archibate

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill automates and operationalizes the end-to-end lifecycle of machine learning systems so teams can eliminate manual orchestration, inconsistent datasets, and brittle deployments that hinder repeatability and reliability.

Core Features & Use Cases

  • Pipeline architecture & orchestration: Design DAGs and component dependencies for Airflow, Dagster, Kubeflow, and Prefect.
  • Data preparation & versioning: Data validation, feature engineering, dataset versioning, and lineage tracking.
  • Training & experiment management: Coordinate training jobs, hyperparameter management, distributed training patterns, and integrate experiment tracking (MLflow, W&B).
  • Validation & deployment automation: Model validation frameworks, A/B testing, canary and blue-green releases, rollback strategies, and production monitoring.
  • Use Case: Implement a reproducible pipeline that versions raw and processed data, runs scheduled training with experiment tracking, and deploys validated models to Kubernetes with canary rollouts and monitoring.

Quick Start

Create an end-to-end ML pipeline that ingests my dataset, runs validation and feature engineering, trains and tracks experiments, and deploys a canary model to Kubernetes.

Frequently Asked Questions about ml-pipeline-workflow

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build an end-to-end ML pipeline with Airflow and MLflow?▼

Build an end-to-end ML pipeline by designing DAGs for data ingestion, training, and deployment, while integrating MLflow for experiment tracking. This automates validation, feature engineering, and orchestration.

What is the best way to orchestrate reproducible training workflows in Kubernetes?▼

Orchestrate reproducible training workflows by defining DAG-based dependencies and dataset versioning. Deploying validated models to Kubernetes with canary rollouts ensures reliable production serving and monitoring.

How do I automate model validation and canary rollouts in an MLOps pipeline?▼

Automate model validation and canary rollouts by embedding validation frameworks and deployment strategies directly into your pipeline orchestration. This enables automated A/B testing, blue-green releases, and production monitoring.

Does this pipeline orchestration approach support data versioning and lineage tracking?▼

Yes, pipeline orchestration supports data versioning and lineage tracking. It automates data validation, feature engineering, and dataset versioning to eliminate inconsistent datasets and ensure reproducible training jobs.

Can I use Kubeflow for distributed training patterns and experiment tracking?▼

Yes, you can use Kubeflow to coordinate distributed training patterns and manage hyperparameters. It integrates seamlessly with experiment tracking tools like MLflow and W&B within your DAG-based orchestration.

When do I need DAG-based orchestration for machine learning deployment?▼

You need DAG-based orchestration for machine learning deployment when automating complex component dependencies across data ingestion, training, and serving. It eliminates manual orchestration and prevents brittle, inconsistent deployments.

Related Skills

Explore More Data & Analytics Skills→
dominodatalabdominodatalab
6

domino-flows

Orchestrate multi-step machine learning workflows as DAGs with typed inputs and outputs.

Official
Advanced
tranhieutttranhieutt
70

ml-engineer

Build production ML systems with PyTorch and TensorFlow for serving and monitoring.

Community
Advanced
Sumeet138Sumeet138
1

ml-pipeline-workflow

Orchestrate end-to-end ML pipelines from data ingestion to deployment.

Community
Advanced
FrekiManagarmFrekiManagarm

source-command-sparc-devops

Automate DevOps workflows for deployment, provisioning, configuration, monitoring, and cleanup.

Community
Advanced