databricks-python-sdk

Provides reference guidance and code examples for the Databricks Python SDK, Connect, CLI, and REST API.

Updated Mar 5, 2026
One-click install
npx skills add https://github.com/FMurray/mlfts --skill databricks-python-sdk-fmurray
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: databricks-python-sdk
Source: https://github.com/FMurray/mlfts/tree/main/.agents/skills/databricks-python-sdk
Command: npx skills add https://github.com/FMurray/mlfts --skill databricks-python-sdk-fmurray

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires databricks-sdk, databricks-connect.

What problem does it solve? Writing code against Databricks services requires knowing the correct SDK methods, authentication patterns, and API signatures across clusters, jobs, SQL warehouses, Unity Catalog, model serving, and vector search. This Skill supplies verified method signatures, documentation URLs, and working examples so you avoid guessing parameters or misusing synchronous SDK calls in async applications. ## Core Features & Use Cases - Complete API Reference: Covers WorkspaceClient and AccountClient operations for clusters, jobs, SQL statement execution, warehouses, Unity Catalog (catalogs, schemas, tables, volumes), files, serving endpoints, vector search, pipelines, secrets, and DBUtils, each mapped to official documentation URLs. - Authentication Patterns: Documents PAT tokens, OAuth, Azure Service Principal, named profiles, and environment variable configuration. - Async Safety Guidance: Explains the critical requirement to wrap blocking SDK calls with asyncio.to_thread in FastAPI and asyncio applications. - Use Case: When building a FastAPI service that queries a SQL warehouse and reads Unity Catalog tables, activate this Skill to get correct execute_statement parameters, pagination handling, and non-blocking async patterns. ## Quick Start Ask the agent to write Python code using the Databricks SDK to list clusters and run a SQL query on a warehouse, and it will apply the correct authentication and wait patterns.

Frequently Asked Questions about databricks-python-sdk

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I authenticate the Databricks Python SDK?

Set DATABRICKS_HOST and DATABRICKS_TOKEN environment variables, then call WorkspaceClient() with no arguments. Alternatives include passing host and token explicitly, using a named profile from ~/.databrickscfg, or configuring an Azure Service Principal with tenant and client credentials.

How do I run SQL queries with the Databricks SDK?

Use w.statement_execution.execute_statement with a warehouse_id, SQL statement, and wait_timeout. Check response.status.state for SUCCEEDED, then read rows from response.result.data_array. For large results, fetch additional chunks with get_statement_result_chunk_n.

Does the Databricks SDK work with FastAPI or asyncio?

The Databricks SDK is fully synchronous and blocks the event loop in async applications. Wrap every SDK call with asyncio.to_thread, for example await asyncio.to_thread(lambda: list(w.clusters.list())), to keep FastAPI endpoints responsive.

When should I use api_client.do instead of SDK methods?

Use w.api_client.do for direct REST API calls when an endpoint is not yet exposed in the SDK or when the SDK abstraction is problematic. Prefer typed SDK methods when available since they provide structured responses and error handling.

How do I wait for long-running Databricks operations like cluster creation?

Use the *_and_wait variants such as w.clusters.create_and_wait with a timeout, or call .result() on the Wait object returned by non-blocking methods. You can also poll with callbacks using methods like wait_get_cluster_running.

What is the difference between Databricks Connect and the Databricks SDK?

Databricks Connect runs Spark code locally against a remote cluster via DatabricksSession, while the Databricks SDK manages workspace resources like clusters, jobs, and tables through REST APIs. Install databricks-connect for Spark operations and databricks-sdk for resource management.