skills.rest
Design-System-ET
Design-System-ETOFFICIAL
Explore all skills by this author →
GitHub

ml-pipeline

Design Kubeflow or Airflow DAGs for ML training workflows with MLflow tracking.

Updated Jun 16, 2026
Search Tags:#mlops#machine learning#orchestration#pipeline#mlflow#kubeflow
One-click install
npx skills add https://github.com/Design-System-ET/genexus-dev-opencode --skill ml-pipeline-design-system-et
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ml-pipeline
Source: https://github.com/Design-System-ET/genexus-dev-opencode/tree/main/skills/ml-pipeline
Command: npx skills add https://github.com/Design-System-ET/genexus-dev-opencode --skill ml-pipeline-design-system-et

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires mlflow, scikit-learn, kfp, pandas, great_expectations, wandb, and includes references (resource) components.

What problem does it solve?

This Skill addresses the complexity of building and maintaining production-ready machine learning infrastructure, ensuring that training workflows are reproducible, scalable, and properly monitored.

Core Features & Use Cases

  • Pipeline Orchestration: Create robust DAGs for training workflows using Kubeflow or Airflow.
  • Experiment Tracking: Integrate MLflow or Weights & Biases to log parameters, metrics, and model artifacts.
  • Data Validation: Implement automated schema and distribution checks using Great Expectations to prevent training on corrupted data.
  • Use Case: When deploying a new recommendation model, use this Skill to define a pipeline that validates incoming user data, trains the model with hyperparameter tuning, logs the results to MLflow, and registers the model only if it passes evaluation thresholds.

Quick Start

Use the ml-pipeline skill to design a Kubeflow DAG that orchestrates data validation, model training with MLflow logging, and automated deployment for my churn prediction model.

Frequently Asked Questions about ml-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a production-grade ML pipeline with Kubeflow and MLflow?▼

Build a production-grade ML pipeline by designing Kubeflow DAGs for training orchestration and integrating MLflow to automatically log parameters, metrics, and model artifacts for reproducible lifecycle management.

What's the best way to prevent training on corrupted data in an ML pipeline?▼

Prevent training on corrupted data by implementing automated schema and distribution checks using Great Expectations, establishing robust data validation checkpoints before model training begins.

Does this ML pipeline orchestration approach work with scikit-learn and pandas?▼

Yes, this orchestration approach works with scikit-learn and pandas, integrating these dependencies to handle data processing and model training within containerized environments for scalable execution.

Can I use Weights & Biases for experiment tracking instead of MLflow?▼

Yes, you can use Weights & Biases as an alternative to MLflow for experiment tracking, logging parameters and metrics to monitor training workflows and ensure model reproducibility.

How do I automate model deployment after hyperparameter tuning and evaluation?▼

Automate model deployment by defining a pipeline that trains with hyperparameter tuning, logs results, and registers the model to the registry only if it passes strict evaluation thresholds.

When do I need data validation checkpoints in machine learning pipelines?▼

You need data validation checkpoints in machine learning pipelines when deploying models into production, ensuring incoming data conforms to schema and distribution requirements before training.

Related Skills

Explore More Software Engineering Skills→
pluginagentmarketplacepluginagentmarketplace
1

devops-patterns

Provide repeatable DevOps patterns for Docker, Kubernetes, Terraform, and CI/CD workflows.

Community
Advanced
ouakarouakar

ai-engineer

Design and deploy production AI/ML systems with MLOps pipelines and monitoring.

Community
Advanced
ArdurAIArdurAI
1

god-data-cleaning

Automate machine learning data preparation with cleaning, preprocessing, feature engineering, and validation.

Official
Advanced
drgaciwdrgaciw

ml-pipeline-workflow

Build end-to-end MLOps pipelines for model training and deployment.

Community
Advanced