databricks-zerobus-ingest

Ingest records into Databricks Delta tables via Zerobus Ingest gRPC.

27|9|Updated Feb 3, 2026

Search Tags:#sdk #ingestion #protobuf #databricks #delta-lake #ack #zerobus-ingest

Ready to run

npx skills add https://github.com/datasciencemonkey/coding-agents-databricks-apps --skill databricks-zerobus-ingest

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-zerobus-ingest
Source: https://github.com/datasciencemonkey/coding-agents-databricks-apps/tree/main/.claude/skills/databricks-zerobus-ingest
Command: npx skills add https://github.com/datasciencemonkey/coding-agents-databricks-apps --skill databricks-zerobus-ingest

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Zerobus Ingest enables direct, serverless ingestion of records into Databricks Delta tables via gRPC, removing the need for an external message bus and enabling near-real-time data delivery.

Core Features & Use Cases

Protobuf and JSON ingestion across multiple languages via the Zerobus Ingest SDKs (Python, Java, Go, TypeScript, Rust)
Protobuf schema generation from Unity Catalog tables and type-safe bindings for generated records
Stream-based ingestion with ACK handling, retries, and backoff to ensure durability
Pre-create and manage target Delta tables; monitor throughput, and implement schema evolution with re-generation of .proto files

Quick Start

Install the Zerobus Ingest SDK and configure your environment to begin streaming data into your Delta table.

Frequently Asked Questions about databricks-zerobus-ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I ingest records directly into Databricks Delta tables without an external message bus?▼

Direct Delta ingestion into Databricks without an external message bus is enabled via Zerobus Ingest gRPC for near-real-time serverless data delivery. It streams records directly into Unity Catalog managed tables using ACK handling and retry logic.

What programming languages are supported by the Zerobus Ingest SDK for Delta Lake ingestion?▼

Delta Lake ingestion via the Zerobus Ingest SDK supports Python, Java, Go, TypeScript, and Rust. These SDKs facilitate Protobuf and JSON record streaming with type-safe bindings generated from Unity Catalog schemas.

How do I generate Protobuf schemas from Databricks Unity Catalog for stream ingestion?▼

Protobuf schema generation from Unity Catalog extracts table definitions to create .proto files for stream ingestion. Regenerating these files handles schema evolution and provides type-safe bindings for generated records across supported SDK languages.

What prerequisites are needed to stream data into a Delta table using gRPC?▼

Streaming data into a Delta table via gRPC requires a Unity Catalog managed Delta table and a service principal with MODIFY and SELECT permissions. The Zerobus Ingest SDK must also be installed in your target development environment.

How does stream-based ingestion handle delivery durability and retries?▼

Stream-based ingestion handles delivery durability through ACK handling, automated retries, and backoff logic. This ensures records sent over gRPC are successfully committed to the Delta table, preventing data loss during transient failures.