databricks-zerobus-ingest

Ingest data into Databricks Delta tables via gRPC with retry logic.

11|3|Updated Jun 10, 2025

Search Tags:#sdk #protobuf #delta #grpc #ingest #databricks #zerobus

Ready to run

npx skills add https://github.com/Paldom/databricks-apps-fastapi-starter --skill databricks-zerobus-ingest-paldom

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-zerobus-ingest
Source: https://github.com/Paldom/databricks-apps-fastapi-starter/tree/main/.gemini/skills/databricks-zerobus-ingest
Command: npx skills add https://github.com/Paldom/databricks-apps-fastapi-starter --skill databricks-zerobus-ingest-paldom

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Zerobus Ingest enables direct, serverless ingestion into Unity Catalog managed Delta tables via gRPC, eliminating the need for a message bus and enabling near real-time data pipelines.

Core Features & Use Cases

Cross-language producers: build ingest clients in Python, Java, Go, TypeScript, and Rust.
Protobuf schema generation from UC tables: automatically generate type-safe schemas for your Delta tables.
Streaming with ACKs and retry logic: durable ingestion with configurable retry strategies and backpressure handling.
Quick-start patterns for prototyping and production: from quick JSON pipelines to protobuf-based production workloads.

Quick Start

Install the Zerobus Ingest SDK, configure your Databricks workspace, create a stream, ingest records, and flush to ensure durability.

Frequently Asked Questions about databricks-zerobus-ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I ingest data directly into Databricks Delta tables without a message bus?▼

You can ingest data directly into Databricks Delta tables using Zerobus Ingest via gRPC. This serverless approach eliminates the need for a message bus, enabling near real-time data pipelines into Unity Catalog managed tables.

Can I build Delta ingestion producers in multiple programming languages?▼

Yes, Zerobus Ingest supports cross-language producers for Delta ingestion. You can build ingest clients in Python, Java, Go, TypeScript, and Rust, allowing flexible integration across different technology stacks.

How do I generate type-safe Protobuf schemas from Unity Catalog tables?▼

Zerobus Ingest automatically generates type-safe Protobuf schemas from Unity Catalog tables. This feature allows you to create structured schemas for your Delta tables, ensuring type safety during data ingestion.

Does Zerobus Ingest handle streaming with ACKs and retry logic for Delta ingestion?▼

Yes, Zerobus Ingest provides streaming with ACK handling and retry logic for durable Delta ingestion. It includes configurable retry strategies and backpressure handling to ensure production readiness for your data pipelines.

What prerequisites do I need to set up before ingesting data into Unity Catalog Delta tables?▼

You need the Zerobus Ingest SDK, gRPC connectivity, and pre-created Unity Catalog Delta tables with explicit access grants. You must configure your Databricks workspace, create a stream, and flush records to ensure durability.