Design and implement Databricks data pipelines with Unity Catalog governance.
npx skills add https://github.com/kumewata/dotfiles --skill databricks
Please help me install this Agent Skill. Skill: databricks Source: https://github.com/kumewata/dotfiles/tree/main/config/agents/skills/databricks Command: npx skills add https://github.com/kumewata/dotfiles --skill databricks
Provides a structured, hands-on guide to building and managing Databricks-based data pipelines, governance, and ML infrastructure.
Initialize a Databricks project by following the steps in this guide to authenticate, configure a warehouse, and start a sample job.
High-intent search queries and answers about installing and using this skill.
Databricks CLI authentication setup involves configuring workspace credentials and initializing a project to manage warehouses via the SQL API. This process enables automated pipeline execution and secure infrastructure access.
Designing Unity Catalog permissions requires structuring access controls across the 3-level namespace of catalog, schema, and table. This governance model secures data assets while enabling scalable pipeline workflows.
Building Delta Lake pipelines with Lakeflow involves configuring scalable data workflows that handle batch and streaming data. This integration ensures reliable data engineering pipelines across Databricks workspaces.
Deploying MLflow models involves configuring model serving patterns and integrating the feature store for consistent training data. This ML infrastructure setup streamlines scalable model deployment across workspaces.
Implementing ML infrastructure requires familiarity with Databricks workspaces, CLI usage, and Unity Catalog concepts. Entry barriers include understanding scalable pipeline orchestration and permission design.
Unity Catalog provides centralized governance through its 3-level namespace, distinguishing it from alternative methods. It natively integrates with Lakeflow and Delta Lake to secure and manage scalable data pipelines.
wesleyegbertoProvision and validate hybrid cloud network topologies with VPN and dedicated interconnects.
AradotsoCreate interactive Power BI sales dashboards with predictive forecasting.
neo4j-contribRead and write Neo4j data with Apache Spark DataFrames.
avdelag1Design end-to-end ML pipelines with Kubeflow, MLflow, and DVC.
