databricks-sql

Generate Databricks SQL queries with Spark-specific functions and Delta Lake Time Travel.

1|Updated Jan 26, 2026

Search Tags:#query optimization #sql #spark #databricks #data warehousing #delta lake

Ready to run

npx skills add https://github.com/olavocarvalho/data-agents --skill databricks-sql-olavocarvalho

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-sql
Source: https://github.com/olavocarvalho/data-agents/tree/main/skills/databricks-sql
Command: npx skills add https://github.com/olavocarvalho/data-agents --skill databricks-sql-olavocarvalho

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill enables users to write advanced Databricks SQL queries, leveraging Spark-specific features that go beyond standard SQL for complex data manipulation and analysis.

Core Features & Use Cases

Spark-Specific Functions: Utilize higher-order functions, array/map operations, and VARIANT for semi-structured data.
Query Optimization: Use EXPLAIN to analyze query execution plans and QUALIFY to filter window function results efficiently.
Delta Lake Time Travel: Query historical data versions for auditing and debugging.
Use Case: Analyze historical sales data by querying a specific past version of a Delta Lake table using Time Travel, or optimize a complex query by examining its execution plan with EXPLAIN.

Quick Start

Use the databricks-sql skill to write a query that selects the latest order for each customer using QUALIFY and ROW_NUMBER.

Frequently Asked Questions about databricks-sql

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I optimize Databricks SQL queries using QUALIFY and EXPLAIN?▼

To optimize Databricks SQL queries, use the QUALIFY clause to filter window function results efficiently and run EXPLAIN to analyze the query execution plan for identifying performance bottlenecks.

What is the best way to query historical data in a Delta Lake table?▼

Querying historical data in a Delta Lake table is best achieved using Time Travel, which allows you to specify and retrieve data from a specific past version for auditing and debugging purposes.

Can I use higher-order functions for array and map operations in Spark SQL?▼

Yes, you can use higher-order functions in Spark SQL to perform concise and advanced data transformations directly on complex arrays and maps within your Databricks queries.

How do I handle semi-structured data like JSON and VARIANT in Databricks?▼

Handle semi-structured data like JSON and VARIANT in Databricks by using Spark-specific SQL functions designed to parse, query, and manipulate nested schemas efficiently.

Do I need standard SQL knowledge to use Databricks SQL features?▼

Standard SQL knowledge is required, but you must also learn Spark-specific features like QUALIFY, Time Travel, and higher-order functions to fully leverage Databricks SQL for complex data manipulation.