skills.rest
prashsub
prashsubCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

ml-pipeline-setup

Orchestrate Databricks ML pipelines with MLflow and Unity Catalog.

1|Updated Jan 22, 2026
Search Tags:#mlops#machine learning#feature engineering#mlflow#databricks#unity catalog#batch inference
One-click install
npx skills add https://github.com/prashsub/vibe_coding_lakehouse_starter_repo --skill ml-pipeline-setup-prashsub
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ml-pipeline-setup
Source: https://github.com/prashsub/vibe_coding_lakehouse_starter_repo/tree/main/data_product_accelerator/skills/ml/00-ml-pipeline-setup
Command: npx skills add https://github.com/prashsub/vibe_coding_lakehouse_starter_repo --skill ml-pipeline-setup-prashsub

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires databricks-asset-bundles, databricks-python-imports, naming-tagging-standards, databricks-autonomous-operations, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill streamlines the creation, training, and deployment of production-grade Machine Learning pipelines on Databricks, ensuring consistency and reducing debugging time.

Core Features & Use Cases

  • End-to-End ML Workflow: Covers feature engineering, model training, and batch inference.
  • MLflow & Unity Catalog Integration: Leverages best practices for tracking, model registry, and data governance.
  • Use Case: Implement a new ML model for cost anomaly detection, from feature creation to batch scoring, following established patterns for reliability and maintainability.

Quick Start

Use the ml-pipeline-setup skill to create feature tables, train all defined models, and run batch inference jobs.

Frequently Asked Questions about ml-pipeline-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up an end-to-end machine learning pipeline on Databricks for batch inference?▼

Setting up an ML pipeline on Databricks involves orchestrating feature engineering, model training, and batch inference workflows while integrating MLflow for tracking and Unity Catalog for data governance.

What are the best practices for MLflow experiment tracking and model registration in Databricks?▼

Best practices for MLflow tracking involve following 19 non-negotiable rules for experiment tracking, data logging, and model registration to ensure production-grade reliability and reduce debugging time.

Does this ML pipeline setup integrate with Unity Catalog for data governance?▼

Yes, the ML pipeline setup integrates with Unity Catalog to manage feature tables and provide robust data governance alongside MLflow model registry operations.

How do I automate feature engineering and model training workflows in Databricks?▼

Automate feature engineering and model training by using the provided scripts and assets to define workflows, which handle data logging and error handling according to strict MLOps standards.

What Databricks dependencies are required to run production-grade MLOps pipelines?▼

Running MLOps pipelines requires Databricks Asset Bundles, specific Python imports, naming and tagging standards, and autonomous operations components to manage the workflow execution environment.

Related Skills

Explore More Data & Analytics Skills→
Zack2626-okZack2626-ok

databricks-bundles

Automate creation, configuration, and deployment of Databricks Asset Bundles.

Community
Advanced
databricks-solutionsdatabricks-solutions
5

ml-pipeline-setup

Orchestrate end-to-end ML pipelines on Databricks with MLflow and Unity Catalog.

Official
Advanced
tja688tja688

unity-catalog-tagger

Apply and manage Unity Catalog metadata tags for tables, columns, and schemas.

Community
Advanced
YOYOMAIIYOYOMAII
1

clerk-custom-ui

Implement custom authentication flows and style Clerk components in Next.js applications.

Community
Advanced