transactional-outbox

To publish events to Kafka atomically with database writes, use the transactional outbox pattern to persist business data and an outbox record in a single transaction, then use Debezium CDC to reliably route outbox changes to Kafka.

The transactional outbox pattern solves atomicity gaps by saving business data and an event record in one database transaction. Debezium CDC captures outbox table changes and streams them to Kafka, ensuring reliable event-driven messaging without lost or ghost messages.

Implement the outbox pattern by creating an outbox table with a matching JPA entity to persist business data and events in a single transaction. Configure a Debezium connector to capture outbox changes and stream them to Kafka for your microservices.

The best way to ensure idempotency with a transactional outbox is to implement idempotent consumers and recovery scenarios. This maintains exactly-once-ish semantics by safely handling duplicate event delivery across your streaming pipelines.

Yes, you can use an optional polling publisher instead of Debezium CDC. It acts as a deterministic fallback mechanism for event delivery, polling the outbox table to publish events to Kafka when change data capture is unavailable.

You should not use Debezium CDC if your database does not support change data capture or if you need a simpler deterministic fallback. In these cases, implement an optional polling publisher to read the outbox table and publish events to Kafka.