databricks-vector-search

Create and query Databricks Vector Search endpoints and indexes for RAG applications.

4|1|Updated May 22, 2026

Search Tags:#semantic search #embeddings #rag #vector search #hybrid search #databricks #delta sync

Ready to run

npx skills add https://github.com/ThomazRossito/ai-data-agents --skill databricks-vector-search-thomazrossito

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-vector-search
Source: https://github.com/ThomazRossito/ai-data-agents/tree/main/plugins/ai-data-agents/skills/databricks-vector-search
Command: npx skills add https://github.com/ThomazRossito/ai-data-agents --skill databricks-vector-search-thomazrossito

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires databricks-sdk, databricks-vectorsearch, and includes references (resource) components.

What problem does it solve? Building semantic search and RAG pipelines on Databricks requires choosing between endpoint types, index types, embedding strategies, and filter syntaxes, and mistakes lead to slow queries, stale indexes, or failed syncs. ## Core Features & Use Cases - Endpoint and Index Management: Create Standard or Storage-Optimized endpoints and Delta Sync or Direct Access indexes with managed or self-managed embeddings. - Query Patterns: Run ANN, hybrid, and full-text searches with endpoint-appropriate filters (dict-based for Standard, SQL-like for Storage-Optimized). - End-to-End RAG: Follow a complete walkthrough from source Delta table to index sync to wiring a VectorSearchRetrieverTool into a ChatAgent on Model Serving. - Use Case: Index a knowledge base Delta table with managed embeddings, trigger a sync, and query it with hybrid search to power a support chatbot that matches both meaning and exact error codes. ## Quick Start Use the databricks-vector-search skill to create a Storage-Optimized endpoint and a Delta Sync index over my documents table, then run a hybrid search query.

Frequently Asked Questions about databricks-vector-search

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I create a Databricks Vector Search index from a Delta table?▼

Create a Delta Sync index with w.vector_search_indexes.create_index, specifying the source table, primary key, and an embedding source column pointing to a model like databricks-gte-large-en. Then trigger a sync and wait for the index to reach ONLINE state.

What is the difference between Standard and Storage-Optimized vector search endpoints?▼

Standard endpoints offer 20-50ms query latency and hold about 320M vectors, while Storage-Optimized endpoints cost roughly 7x less, index 20x faster, and scale past 1B vectors at 300-500ms latency. Storage-Optimized is the default choice unless sub-100ms latency is required.

When should I use hybrid search instead of ANN in Databricks?▼

Use hybrid search when queries contain exact terms that must match, such as SKUs, error codes, or proper nouns, since it combines vector similarity with BM25 keyword scoring. Start with ANN for conceptual queries and switch to HYBRID if relevant documents are missed.

Why is filters_json not working on my vector search query?▼

Filter syntax depends on endpoint type: Standard endpoints use dict-format filters via filters_json, while Storage-Optimized endpoints use SQL-like string filters through the databricks-vectorsearch package's filters parameter. Using the wrong format silently has no effect.

Why is my Databricks vector index not updating after table changes?▼

Indexes with TRIGGERED pipeline type require a manual sync_index call after the source Delta table changes. Only CONTINUOUS pipelines auto-sync, and calling sync on a CONTINUOUS index raises an error.

Can I change a vector search endpoint type after creation?▼

No, endpoints are immutable after creation. To migrate, create a new endpoint with the desired type, recreate the indexes on it, wait for sync, update applications to the new index names, then delete the old resources.