databricks-spark-declarative-pipelines

This Skill automates the creation of Databricks Lakeflow Spark Declarative Pipelines using serverless compute. It manages streaming tables, materialized views, and Auto Loader ingestion patterns through Databricks Asset Bundles or MCP tools.

Yes, you can implement SCD Type 2 and CDC in Databricks Lakeflow pipelines. This Skill facilitates these data transformations alongside streaming tables and materialized views, automating the configuration of declarative pipelines using serverless compute.

The best way to automate Databricks data pipeline creation is using Databricks Asset Bundles. This Skill initializes Spark Declarative Pipeline projects and automates their lifecycle management, configuration, and updates via MCP tools.

Yes, it supports Databricks Asset Bundles for project initialization. You can use the databricks pipelines init command to create a new Spark Declarative Pipeline project structure, define the pipeline, and deploy it to Databricks.

You manage streaming tables and materialized views in Databricks SDP by defining them within your declarative pipeline configuration. This Skill automates the setup and updating of these structures on serverless compute.