skills.rest
AngelP17
AngelP17COMMUNITY
Explore all skills by this author →
WebsiteGitHub

ml-pipeline-workflow

Automate end-to-end ML pipelines from data ingestion to deployment.

1|Updated Jan 29, 2026
Search Tags:#orchestration#Airflow#Kubeflow#ml-pipeline#DAG#model-deployment#experiment-tracking
One-click install
npx skills add https://github.com/AngelP17/factoryops-console --skill ml-pipeline-workflow-angelp17
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ml-pipeline-workflow
Source: https://github.com/AngelP17/factoryops-console/tree/main/factoryops-console/.agent/skills/ml-pipeline-workflow
Command: npx skills add https://github.com/AngelP17/factoryops-console --skill ml-pipeline-workflow-angelp17

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides a structured, production-ready blueprint to build, orchestrate, and automate end-to-end ML pipelines, from data ingestion and preparation to model training, validation, deployment, and monitoring.

Core Features & Use Cases

  • Pipeline Architecture: End-to-end workflow design and DAG orchestration across Airflow, Dagster, Kubeflow.
  • Data Preparation: Validation, feature engineering, versioning, and data lineage.
  • Model Training: Training job orchestration, hyperparameter management, experiment tracking.
  • Model Validation: Validation metrics, A/B testing, regression detection.
  • Deployment Automation: Serving patterns, canary/blue-green deployments, rollback mechanisms.
  • Use cases include building production ML workflows, designing reusable pipelines, automating data → train → deploy.

Quick Start

  • Start with a minimal three-stage pipeline: data_ingestion → data_preparation → model_training → model_deployment.
  • Extend with validation, monitoring, and experiment tracking using the references/ and assets/ templates (e.g., references/data-preparation.md, assets/pipeline-dag.yaml.template).
  • Command example: "Create a new ml-pipeline-workflow that ingests data, trains a model, and deploys it to a serving endpoint with basic validation."

Frequently Asked Questions about ml-pipeline-workflow

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I orchestrate an end-to-end ML pipeline from data ingestion to model deployment?▼

Orchestrate ML pipelines using DAG-based tools like Airflow, Dagster, or Kubeflow to automate workflows across data ingestion, preparation, training, validation, and deployment stages. This Skill provides production-ready templates and architecture patterns for each stage, reducing manual handoff complexity.

Can I use Airflow or Kubeflow to manage experiment tracking and model validation in my ML workflow?▼

Yes. Both Airflow and Kubeflow integrate with experiment tracking and data validation frameworks. This Skill includes orchestration patterns for hyperparameter management, validation metrics, A/B testing, and regression detection within your DAG-based pipeline.

What's the best way to automate model deployment with canary or blue-green strategies?▼

Embed deployment automation into your DAG orchestration with canary and blue-green deployment templates, rollback mechanisms, and serving patterns. This Skill provides deployment templates in assets/ that connect to your orchestrator's validation gates.

Do I need data versioning and lineage tracking for production ML pipelines?▼

Data versioning and lineage tracking are critical for reproducibility and debugging in production ML workflows. This Skill covers data validation, feature engineering, and lineage practices integrated into your orchestration DAG.

How do I design reusable, modular ML pipelines that scale across multiple projects?▼

Build modular pipelines by separating concerns into reusable stages—data preparation, training, validation, deployment—orchestrated via DAGs. This Skill provides reference architectures and templates that support extension and adaptation across projects.

Related Skills

Explore More Data & Analytics Skills→
EngineerWithAIEngineerWithAI
4

ml-pipeline-workflow

Orchestrate end-to-end MLOps pipelines from data preparation to production deployment.

Official
Advanced
KentoShimizuKentoShimizu
7

mlops-monitoring-drift

Detect data drift, concept drift, and quality degradation in production ML pipelines.

Community
Advanced
AJBcodingAJBcoding
4

moai-domain-ml-ops

Orchestrate enterprise ML lifecycle workflows across Kubernetes-based MLOps tools.

Community
Advanced
codeApe-7codeApe-7
17

ml-pipeline

Construct automated ML pipelines with data validation, feature engineering, training, and deployment.

Community
Advanced