skills.rest
pluginagentmarketplace
pluginagentmarketplaceCOMMUNITY
Explore all skills by this author →
GitHub

training-pipelines

Orchestrates end-to-end ML training pipelines with PyTorch DDP, Optima, and Kubeflow.

1|Updated Nov 18, 2025
Search Tags:#PyTorch#Kubeflow#orchestrate#distributed-training#hyperparameter-tuning#training-pipelines#Optuna
One-click install
npx skills add https://github.com/pluginagentmarketplace/custom-plugin-mlops --skill training-pipelines
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: training-pipelines
Source: https://github.com/pluginagentmarketplace/custom-plugin-mlops/tree/main/skills/training-pipelines
Command: npx skills add https://github.com/pluginagentmarketplace/custom-plugin-mlops --skill training-pipelines

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires PyYAML, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

Training pipelines simplify the creation, orchestration, and deployment of end-to-end machine learning training workflows, reducing setup time and avoiding boilerplate.

Core Features & Use Cases

  • End-to-end orchestration: Define and manage data loading, preprocessing, training, evaluation, and model registration in a reproducible pipeline.
  • Distributed training: Support for multi-GPU and distributed training using PyTorch DDP with proper data sharding and synchronization.
  • Hyperparameter tuning: Integrates with Optuna to explore configurations and find optimal models.
  • Kubeflow deployment: Provides templates to deploy pipelines to Kubeflow or similar orchestration platforms.
  • Real-world use case: A team trains multiple experiments across GPU clusters with automated validation and artifact storage.

Quick Start

Use the training-pipelines skill to spin up a simple training workflow on a GPU-enabled environment. For example: claude "training-pipelines - [describe a task]"

Frequently Asked Questions about training-pipelines

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I orchestrate distributed training with PyTorch DDP on Kubeflow?▼

PyTorch DDP orchestration on Kubeflow uses templates to manage multi-GPU clusters, applying data sharding and synchronization for distributed training workflows.

How do I set up hyperparameter tuning in a PyTorch training pipeline?▼

Hyperparameter tuning in PyTorch training pipelines integrates Optuna to explore configurations and find optimal models during the automated training workflow.

How do I deploy machine learning training pipelines to GPU clusters?▼

Deploying ML training pipelines to GPU clusters uses Kubeflow templates for production deployment, managing GPU resources and automating validation across experiments.

Can I use Optuna with Kubeflow for automated model training?▼

Yes, Optuna integrates with Kubeflow deployment templates to orchestrate hyperparameter tuning and automate validation across distributed GPU cluster experiments.

What is the best way to manage multi-GPU training workflows?▼

Managing multi-GPU training workflows uses PyTorch DDP setup with proper data sharding and synchronization orchestrated through reproducible end-to-end pipelines.

Do I need PyYAML to define training pipeline templates?▼

Yes, PyYAML is required to define and manage the template configurations for orchestrating end-to-end machine learning training workflows on Kubeflow.

Related Skills

Explore More Software Engineering Skills→
LeonChaoXLeonChaoX
783

pytorch-lightning

Organizes PyTorch training into modules and a managed workflow.

Community
Intermediate
gizixgizix
2

model-deployment

Package trained machine learning models with versioning, ONNX export, and FastAPI endpoints.

Community
Advanced
Supporter09Supporter09

pytorch-lightning

Convert PyTorch models into LightningModule and Trainer workflows.

Community
Advanced
Agentic-AssetsAgentic-Assets

ml-pipeline

Orchestrate end-to-end ML pipelines from data ingestion to deployment.

Official
Advanced