skills.rest
Blackkadder
BlackkadderCOMMUNITY
Explore all skills by this author →
GitHub

databricks-zerobus-ingest

Ingest streaming records into Databricks Delta tables via Zerobus gRPC API.

Updated Mar 19, 2026
Search Tags:#streaming#protobuf#delta#grpc#ingest#databricks#zerobus
One-click install
npx skills add https://github.com/Blackkadder/databricks-apps-and-agents-workshop --skill databricks-zerobus-ingest-blackkadder
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-zerobus-ingest
Source: https://github.com/Blackkadder/databricks-apps-and-agents-workshop/tree/main/.claude/skills/databricks-zerobus-ingest
Command: npx skills add https://github.com/Blackkadder/databricks-apps-and-agents-workshop --skill databricks-zerobus-ingest-blackkadder

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Zerobus Ingest removes the need for an external message bus by enabling producers to write records directly into Databricks managed Delta tables with durability acknowledgments, reducing operational complexity and lowering end-to-end ingestion latency.

Core Features & Use Cases

  • Direct gRPC ingestion: Create streams and write records (JSON or Protobuf) directly to Unity Catalog managed Delta tables with ACK-based durability.
  • Schema-first Protobuf workflow: Generate .proto from Unity Catalog, compile language bindings, and use typed messages for production-grade ingestion.
  • Operational patterns: Built-in guidance for authentication, explicit table grants for service principals, retry and reconnection strategies, throughput scaling, and region constraints for production systems.

Quick Start

Initialize the Zerobus SDK with your workspace endpoint and service principal, generate the Protobuf schema for your target managed Delta table, and run the provided ingest client to push records and confirm durability via ACKs.

Frequently Asked Questions about databricks-zerobus-ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I ingest streaming data directly into Databricks Delta tables without an external message bus?▼

You can use Zerobus gRPC ingestion to write streaming records directly into Unity Catalog managed Delta tables with durability acknowledgments, eliminating the need for an external message bus and lowering end-to-end latency.

Can I use Protobuf schemas to write records to Delta tables over gRPC?▼

Yes, Zerobus ingestion supports a schema-first Protobuf workflow by generating .proto files from Unity Catalog table metadata, compiling language bindings, and using typed messages for production-grade JSON or Protobuf record ingestion.

What service principal grants are required for writing to Unity Catalog managed tables via gRPC?▼

Your service principal requires explicit MODIFY and SELECT grants on the target Unity Catalog managed Delta tables to authenticate and write streaming records successfully via the Zerobus gRPC API.

Does Zerobus ingestion handle retry and reconnection logic for near-real-time pipelines?▼

Yes, Zerobus ingestion includes built-in operational patterns for retry and reconnection strategies, throughput scaling, ACK handling, and region constraints to ensure durable record delivery in near-real-time pipelines.

Which programming languages are supported for building Zerobus ingestion clients?▼

Zerobus ingestion supports producer clients in Python, Java, Go, TypeScript, and Rust for building near-real-time ingestion pipelines that write streaming records to Databricks Delta tables.

What is the best way to ensure durability when ingesting streaming records into Databricks?▼

The best way to ensure durability is using Zerobus gRPC ingestion with ACK-based durability acknowledgments, which confirms that streaming records are successfully written and committed to managed Delta tables.

Related Skills

Explore More Software Engineering Skills→
JingyiWngJingyiWng

databricks-zerobus-ingest

Ingest records into Databricks Delta tables via the Zerobus gRPC API.

Community
Advanced
juanlamadrid20juanlamadrid20

databricks-zerobus-ingest

Ingest data into Databricks Delta tables via the Zerobus gRPC API.

Community
Advanced
datasciencemonkeydatasciencemonkey
27

databricks-zerobus-ingest

Ingest records into Databricks Delta tables via Zerobus Ingest gRPC.

Community
Advanced
vignesh2027vignesh2027
6

realtime-data-agent

Design and deploy Kafka or Kinesis streaming pipelines with Spark or Flink transforms.

Community
Advanced