databrickslabsOfficial·15 Agent Skills Included

lakeflow-community-connectors

Build, test, and deploy Databricks data ingestion connectors

Builds custom data connectors that ingest records from external APIs into Databricks Lakeflow pipelines. Automates API research, connector implementation, credential collection, incremental sync validation, and pipeline deployment. Eliminates manual Spark coding, fragile test setups, and slow credential handling with simulator-based testing and guided deployment.

60|116

↗

npx skills add databrickslabs/lakeflow-community-connectors --all -g -y

Available:

Instructs your AI agent on the two-phase connector workflow, file boundaries, testing modes, and which slash commands and skills to route each development step to.

All Skills in This Repository (15)

Pure Emerald Level Indicators

📦 In Repo

databrickslabs

implement-partitioned-connector

Implement partitioned data connectors extending LakeflowConnect with SupportsPartitionedStream or SupportsPartition interfaces.

research-write-api-of-source

Research and document POST/PUT write API endpoints for source systems.

write-back-testing

Write test data to source systems and verify connector reads and deletions.

generate-connector-spec

Generate connector_spec.yaml from Python source code with authentication parameters.

validate-incremental-sync

Validate CDC offset tracking and record filtering in data connectors.

implement-connector

Implement Python data source connectors conforming to the LakeflowConnect interface.

validate-connector-auth

Generate and execute an authentication verification test for a data connector.

build_connector_package

Create a pyproject.toml file and build a source connector as a Python package.

create-connector-document

Generate Markdown user documentation for data connectors from code and API docs.

test-and-fix-connector

Execute and fix connector tests using pytest in a Python virtual environment.

collect-credentials

Collects user credentials via a browser form for a data connector.

authenticate-source

Generate connector specifications, collect credentials, and validate data source authentication.

Official

Intermediate

Frequently Asked Questions

FAQPage Schema

How to install Lakeflow Community Connectors?▼

Run `npx skills add databrickslabs/lakeflow-community-connectors --all -g -y` in your terminal to install all connector development skills globally.

How to build a custom Databricks ingestion connector?▼

Run the /develop-connector command with your source name to research the API, implement the connector, and test it against a simulator without needing credentials.

Do I need live API credentials to develop a connector?▼

No. Phase 1 development runs entirely against an in-process simulator with synthetic data; credentials are only needed in Phase 2 for live record-mode validation.

Does it work with Claude Code and Cursor?▼

Yes. All commands, skills, and subagents under .claude/ are auto-discovered by both Claude Code and Cursor with no extra setup.

How are connectors deployed to Databricks?▼

The /deploy-connector skill builds the connector package, uploads it to a Unity Catalog volume, and creates a managed ingestion pipeline from an interactive spec.