kafka-patterns

Implement a Dead Letter Queue in Spring Kafka using a three-stage workflow: work, retry, and dead topics. Partition these topics and include headers for retry counts to track and isolate failed messages effectively.

Configure reliable Kafka producers and consumers by setting idempotence, acks, and batch parameters for producers, while implementing manual commit strategies and specific listener behaviors for consumers to ensure message processing integrity.

Test Kafka DLQ behavior using Testcontainers by configuring your Spring Kafka project with the provided properties and running containerized integration tests to automatically create topics and verify message retry and dead-letter routing.

Design Kafka topics for a retry workflow by creating separate work, retry, and dead topics. Apply partitioning across these topics and attach headers containing retry counts to manage message progression through the retry stages.

Integration testing requires automatic topic creation enabled through Spring configuration. This setup simplifies verifying Kafka consumer and producer behavior alongside Testcontainers without needing manual topic provisioning.

Kafka consumers and producers need idempotence and acks settings to prevent duplicate messages and data loss during network partitions. These configurations ensure reliable message delivery in production-grade Spring Kafka setups.