What problem does it solve?
This Skill removes the operational overhead of managing model endpoints and boilerplate code by enabling AI capabilities (classification, extraction, summarization, translation, redaction, similarity, document parsing, and forecasting) directly inside Databricks SQL and PySpark pipelines so teams can deploy large-scale batch and streaming AI workflows with minimal integration effort.
Core Features & Use Cases
- Task-specific AI Functions: ai_classify, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity for direct column-level enrichment.
- General-purpose and Multimodal: ai_query for complex nested JSON outputs, custom endpoints, and image+text multimodal inputs.
- Document Parsing & RAG: ai_parse_document for PDFs/images/DOCX plus patterns to parse → chunk → index → query for retrieval-augmented generation.
- Time Series Forecasting: ai_forecast as a table-valued function for business forecasting use cases.
- Production Patterns: Centralized config.yml, failOnError handling, DLT/DSPy pipeline examples, batch and streaming designs, and Vector Search integration for RAG systems.
Quick Start
Use ai_parse_document to extract text from PDFs in a Unity Catalog volume, then run ai_summarize and ai_extract to generate summaries and structured fields and write the results to a Delta table.
