databricks-ai-functions

Call Databricks built-in AI Functions directly from SQL and PySpark pipelines.

1|Updated Dec 15, 2025

Search Tags:#nlp #document-processing #data-pipelines #databricks #ai-functions #ai-query #ai-parse-document

Ready to run

npx skills add https://github.com/lucaslessachaves/default --skill databricks-ai-functions-lucaslessachaves

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-ai-functions
Source: https://github.com/lucaslessachaves/default/tree/main/.claude/skills/databricks-ai-functions
Command: npx skills add https://github.com/lucaslessachaves/default --skill databricks-ai-functions-lucaslessachaves

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Databricks AI Functions let teams add AI capabilities directly to SQL and PySpark pipelines without managing external model endpoints, dramatically simplifying AI-powered data processing.

Core Features & Use Cases

Task-specific AI functions for classification, extraction, summarization, translation, parsing, and forecasting
Document ingestion and RAG-style workflows (parse → chunk → index → query)
Clear best-practice patterns and real-world examples for batch and near-real-time pipelines

Quick Start

Run a simple query that classifies text, extracts fields, and summarizes content from a column using ai_classify, ai_extract, and ai_summarize.

Frequently Asked Questions about databricks-ai-functions

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I use AI functions in Databricks SQL and PySpark pipelines without managing external endpoints?▼

You can use Databricks built-in AI functions to integrate AI capabilities directly into SQL and PySpark pipelines, eliminating the need to manage external model endpoints for batch and near-real-time workflows.

Can I build a RAG pipeline in Databricks using built-in AI functions?▼

Yes, Databricks AI functions support RAG-style workflows by providing task-specific functions to parse documents, chunk text, index the content, and query the resulting data directly within your pipelines.

What AI data processing tasks do Databricks built-in AI functions support?▼

Databricks built-in AI functions support task-specific data processing including classification, extraction, summarization, grammar correction, translation, parsing, and forecasting directly within SQL and PySpark.

What is the best way to classify text and extract fields from a column in Databricks?▼

The best way to classify text and extract fields is using the built-in ai_classify and ai_extract functions, which allow you to process column data directly in SQL queries without external model endpoints.

Do I need to configure external model endpoints to run text summarization in Databricks pipelines?▼

No, you do not need external model endpoints. Databricks AI functions like ai_summarize let you perform text summarization directly within SQL and PySpark pipelines using built-in capabilities.

When should I use built-in AI functions versus external models for document processing in Databricks?▼

Use built-in AI functions for document processing when you need to simplify AI-powered data workflows in SQL and PySpark without the operational overhead of managing external model endpoints for batch processing.