databricks-dbsql

You can write stored procedures in Databricks SQL using procedural constructs like CREATE PROCEDURE, CALL, and BEGIN...END blocks to encapsulate complex logic. This enables reliable ETL orchestration and data quality checks directly within the DBSQL environment.

Procedural SQL scripting in Databricks extends standard SQL with variables, loops, and conditionals to build reliable ETL pipelines and governance logic. It eliminates the need for external scripting by handling complex data workflows and exception management natively.

Yes, you can process hierarchical data and perform graph traversals in Databricks using the WITH RECURSIVE clause. This procedural SQL feature allows you to query multi-level hierarchies directly inside the DBSQL environment without external orchestration tools.

Yes, accessing procedural features like EXECUTE IMMEDIATE, MERGE, and WITH RECURSIVE requires Databricks Runtime 17.x or higher. You must also enable Unity Catalog and use serverless SQL warehouses to execute these transactional workflows and automated governance scripts.

You can execute dynamic SQL at runtime in Databricks using the EXECUTE IMMEDIATE statement. This feature generates and runs SQL scripts on the fly, allowing flexible workflow orchestration and automated governance across Delta tables inside BEGIN ATOMIC blocks.

Using procedural SQL instead of external scripting simplifies complex data workflows by keeping transactional controls and exception handling directly inside Databricks. This approach leverages BEGIN ATOMIC blocks for multi-statement reliability and reduces orchestration overhead across Delta tables.