databricks-zerobus-ingest

Ingest records into Unity Catalog Delta tables via serverless gRPC API.

1|Updated Dec 15, 2025

Search Tags:#protobuf #grpc #ingest #delta-lake #unity-catalog #zerobus

Ready to run

npx skills add https://github.com/lucaslessachaves/default --skill databricks-zerobus-ingest-lucaslessachaves

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-zerobus-ingest
Source: https://github.com/lucaslessachaves/default/tree/main/.claude/skills/databricks-zerobus-ingest
Command: npx skills add https://github.com/lucaslessachaves/default --skill databricks-zerobus-ingest-lucaslessachaves

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Zerobus Ingest provides a serverless, direct-to-Delta ingestion path using gRPC, eliminating the need for traditional message buses and enabling low-latency data writes to Unity Catalog-managed Delta tables.

Core Features & Use Cases

Cross-language SDK support (Python, Java, Go, TypeScript, Rust) for streaming ingestion.
JSON and Protobuf serialization with schema validation and durability acknowledgments.
Prototyping to production workflows, schema generation from UC tables, retry and reconnection logic, and end-to-end ingestion with ACK handling.

Quick Start

Create a simple producer that initializes a Zerobus stream and ingests records into a Delta table, followed by a flush to ensure durability.

Frequently Asked Questions about databricks-zerobus-ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I ingest data directly into Delta tables via gRPC?▼

You can ingest data directly into Delta tables via gRPC by using a serverless API to write records into Unity Catalog-managed tables, eliminating the need for traditional message buses. It supports JSON and Protobuf serialization with durability acknowledgments.

What is the best way to stream records into Unity Catalog Delta tables without a message bus?▼

Streaming records into Unity Catalog Delta tables without a message bus is achieved through a serverless gRPC ingestion path. This approach provides low-latency data writes and cross-language SDK support for Python, Java, Go, TypeScript, and Rust.

Do I need a pre-created managed Delta table to use Unity Catalog ingestion?▼

Yes, Unity Catalog ingestion requires a pre-created managed Delta table. You also need a service principal with MODIFY and SELECT grants on that table, and the Zerobus SDK installed in your client environment to establish the gRPC stream.

Can I use Protobuf serialization for near real-time analytics in Delta Lake?▼

Yes, you can use Protobuf serialization for near real-time analytics in Delta Lake. The ingestion API supports both JSON and Protobuf formats, applying schema validation and providing durability acknowledgments for reliable streaming workflows.

How does schema generation from UC tables work for gRPC ingestion?▼

Schema generation from UC tables for gRPC ingestion allows you to automatically derive the data structure required by the serverless API from your existing Unity Catalog tables. This enables schema validation and ensures incoming records match the Delta table format.

What are the limitations of direct-to-Delta gRPC ingestion?▼

Direct-to-Delta gRPC ingestion requires a pre-existing managed table and a configured service principal with specific grants. While it offers low-latency writes, you must handle retry and reconnection logic on the client side to ensure end-to-end durability during network disruptions.