skills.rest
teegin-g
teegin-gCOMMUNITY
Explore all skills by this author →
GitHub

databricks-ai-functions

Run built-in AI inference in SQL and PySpark pipelines.

Updated Jan 30, 2026
Search Tags:#sql#pyspark#document-processing#databricks#ai-functions#ai-parse#ai-query
One-click install
npx skills add https://github.com/teegin-g/Slopcast --skill databricks-ai-functions-teegin-g
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-ai-functions
Source: https://github.com/teegin-g/Slopcast/tree/main/.agents/skills/databricks-ai-functions
Command: npx skills add https://github.com/teegin-g/Slopcast --skill databricks-ai-functions-teegin-g

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Databricks AI Functions enable built-in AI inference directly in SQL and PySpark pipelines without model endpoints or keys. They can run on table columns and support a range of task-specific functions, document parsing, and a general-purpose ai_query path for complex reasoning.

Core Features & Use Cases

  • Task-specific functions for classification, extraction, summarization, translation, and sentiment analysis (ai_classify, ai_extract, ai_summarize, ai_translate, ai_analyze_sentiment, ai_mask, ai_fix_grammar, ai_gen, ai_parse_document).
  • Document ingestion and parsing support for PDFs and office docs via ai_parse_document, with optional AI-driven insights via ai_query for nested JSON or multimodal data.
  • End-to-end pattern guidance for building batch pipelines and RAG workloads (parse → chunk → index → query), with best-practice prompts centralized in config.yml.

Quick Start

Run a sample query to classify support tickets, extract key fields, and summarize the results.

Frequently Asked Questions about databricks-ai-functions

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run AI inference directly in Databricks SQL and PySpark pipelines?▼

Databricks AI Functions enable built-in AI inference directly in SQL and PySpark pipelines without managing model endpoints or keys. You can apply task-specific functions like ai_classify and ai_summarize directly to table columns for batch processing.

Can I parse PDFs and office documents natively in Databricks?▼

Yes, you can parse PDFs and office documents using the ai_parse_document function. It supports document ingestion natively, with optional AI-driven insights available via ai_query for handling nested JSON or multimodal data extraction.

What Databricks Runtime and warehouse types are required for AI Functions?▼

Using Databricks AI Functions requires DBR 15.1 or higher, with optional DBR 17.1+ needed specifically for document parsing. Additionally, running the ai_forecast function requires a Pro or Serverless SQL warehouse.

What is the best way to build a RAG pipeline using Databricks AI Functions?▼

The best way to build a RAG pipeline is following the end-to-end pattern of parse, chunk, index, and query. The skill provides guidance for these batch pipelines, with best-practice prompts centralized in a config.yml file.

What should I use for complex multimodal tasks not covered by specific AI Functions in Databricks?▼

For complex multimodal tasks or nested JSON extraction not covered by task-specific functions, use the general-purpose ai_query path. It serves as a last-resort option to handle complex reasoning within your SQL and PySpark pipelines.

Related Skills

Explore More Data & Analytics Skills→
ThomazRossitoThomazRossito
31

databricks-ai-functions

Run AI functions for parsing, extraction, classification, and forecasting in Databricks SQL and PySpark pipelines.

Community
Advanced
mattwoodcomattwoodco
3

ai-rag-vectors

Create a pgvector embedding layer for semantic search with recursive text chunking.

Community
Advanced
logicalclockslogicalclocks
15

hops-spark

Automate PySpark code execution in Hopsworks with Delta Lake integration.

Official
Advanced
leary-pokenleary-poken
1

databricks-ai-functions

Run AI inference directly in SQL and PySpark pipelines.

Community
Advanced