databricks-zerobus-ingest

Ingest data into Databricks Delta tables via gRPC using the Zerobus Ingest SDK.

Updated Mar 10, 2026

Search Tags:#real-time #protobuf #grpc #ingest #databricks #delta lake #zerobus

Ready to run

npx skills add https://github.com/robkisk/coding-agents-databricks-app --skill databricks-zerobus-ingest-robkisk

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-zerobus-ingest
Source: https://github.com/robkisk/coding-agents-databricks-app/tree/main/.claude/skills/databricks-zerobus-ingest
Command: npx skills add https://github.com/robkisk/coding-agents-databricks-app --skill databricks-zerobus-ingest-robkisk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) and scripts (resource) components.

What problem does it solve?

This Skill enables direct, record-by-record data ingestion into Databricks Delta tables via gRPC, eliminating the need for intermediate message bus infrastructure.

Core Features & Use Cases

Direct Ingestion: Write data directly to Unity Catalog managed Delta tables without Kafka or Kinesis.
SDK Support: Use the Zerobus Ingest SDK in Python, Java, Go, TypeScript, and Rust.
Schema Management: Generate Protobuf schemas from Unity Catalog tables for type-safe ingestion.
Use Case: Build a real-time sensor data pipeline that streams IoT device readings directly into a Delta table for immediate analytics, using Python and Protobuf for reliability.

Quick Start

Use the databricks-zerobus-ingest skill to ingest JSON records into the 'catalog.schema.my_table' table using the provided credentials.

Frequently Asked Questions about databricks-zerobus-ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I ingest data into Delta Lake without using Kafka or Kinesis?▼

You can ingest data directly into Delta Lake without intermediate message buses by using the Zerobus Ingest SDK. This approach streams records via gRPC straight into Unity Catalog managed Delta tables for immediate analytics.

Can I use Python to stream IoT sensor data directly into Databricks Delta tables?▼

Yes, you can stream IoT sensor data into Databricks Delta tables using the Python Zerobus Ingest SDK. It supports building producers that write directly to Unity Catalog tables using Protobuf schemas for type-safe ingestion.

How do I generate Protobuf schemas from Unity Catalog tables for real-time ingestion?▼

The Zerobus Ingest SDK generates Protobuf schemas directly from Unity Catalog tables. This provides type-safe record structures for streaming data into Delta Lake via gRPC connections.

Does the Zerobus Ingest SDK support clients in languages other than Python?▼

Yes, the Zerobus Ingest SDK supports Python, Java, Go, TypeScript, and Rust clients. You can build producers in any of these languages to write data directly to Databricks Delta tables.

How is durability handled during stream-based ingestion into Delta tables?▼

Stream-based ingestion into Delta tables handles durability through acknowledgments. The SDK also implements robust retry and reconnection logic to ensure reliable data delivery in production environments.

What is the best way to implement production-ready retry logic for Databricks gRPC ingestion?▼

The best way to implement reliable retry logic for Databricks gRPC ingestion is using the Zerobus Ingest SDK. It includes built-in reconnection and retry mechanisms designed specifically for production Delta table streaming.