skills.rest
olavocarvalho
olavocarvalhoCOMMUNITY
Explore all skills by this author →
𝕏@olavovieiracomGitHub

databricks-sql

Generate Databricks SQL queries with Spark-specific functions and Delta Lake Time Travel.

1|Updated Jan 26, 2026
Search Tags:#query optimization#sql#spark#databricks#data warehousing#delta lake
One-click install
npx skills add https://github.com/olavocarvalho/data-agents --skill databricks-sql-olavocarvalho
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-sql
Source: https://github.com/olavocarvalho/data-agents/tree/main/skills/databricks-sql
Command: npx skills add https://github.com/olavocarvalho/data-agents --skill databricks-sql-olavocarvalho

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill enables users to write advanced Databricks SQL queries, leveraging Spark-specific features that go beyond standard SQL for complex data manipulation and analysis.

Core Features & Use Cases

  • Spark-Specific Functions: Utilize higher-order functions, array/map operations, and VARIANT for semi-structured data.
  • Query Optimization: Use EXPLAIN to analyze query execution plans and QUALIFY to filter window function results efficiently.
  • Delta Lake Time Travel: Query historical data versions for auditing and debugging.
  • Use Case: Analyze historical sales data by querying a specific past version of a Delta Lake table using Time Travel, or optimize a complex query by examining its execution plan with EXPLAIN.

Quick Start

Use the databricks-sql skill to write a query that selects the latest order for each customer using QUALIFY and ROW_NUMBER.

Frequently Asked Questions about databricks-sql

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize Databricks SQL queries using QUALIFY and EXPLAIN?▼

To optimize Databricks SQL queries, use the QUALIFY clause to filter window function results efficiently and run EXPLAIN to analyze the query execution plan for identifying performance bottlenecks.

What is the best way to query historical data in a Delta Lake table?▼

Querying historical data in a Delta Lake table is best achieved using Time Travel, which allows you to specify and retrieve data from a specific past version for auditing and debugging purposes.

Can I use higher-order functions for array and map operations in Spark SQL?▼

Yes, you can use higher-order functions in Spark SQL to perform concise and advanced data transformations directly on complex arrays and maps within your Databricks queries.

How do I handle semi-structured data like JSON and VARIANT in Databricks?▼

Handle semi-structured data like JSON and VARIANT in Databricks by using Spark-specific SQL functions designed to parse, query, and manipulate nested schemas efficiently.

Do I need standard SQL knowledge to use Databricks SQL features?▼

Standard SQL knowledge is required, but you must also learn Spark-specific features like QUALIFY, Time Travel, and higher-order functions to fully leverage Databricks SQL for complex data manipulation.

Related Skills

Explore More Data & Analytics Skills→
databricks-solutionsdatabricks-solutions
5

gold-layer-setup

Create Databricks Gold layer tables from YAML schema definitions.

Official
Intermediate
ThomazRossitoThomazRossito
31

databricks-execution-compute

Execute Spark, Python, SQL, Scala, and R code on Databricks compute resources.

Community
Advanced
Zack2626-okZack2626-ok

databricks-spark-structured-streaming

Construct real-time data pipelines with Apache Spark Structured Streaming.

Community
Advanced
JonathanMitchell1234JonathanMitchell1234

snowflake-expert

Provide Snowflake SQL examples for warehouse management, data loading, and security.

Community
Advanced