Set up Debezium CDC pipelines exporting database row changes to Kafka.
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill cdc-debezium
Please help me install this Agent Skill. Skill: cdc-debezium Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/skills/cdc_debezium Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill cdc-debezium
This Skill helps you set up and operate reliable Change Data Capture (CDC) pipelines with Debezium so database changes can be streamed to Kafka and downstream sinks without missing updates or duplicating data.
Ask the AI to generate a complete PostgreSQL Debezium connector configuration for tables customers and orders using snapshot.mode initial, ExtractNewRecordState flattening, DLQ enabled, and Iceberg upsert semantics for a lakehouse sink.
High-intent search queries and answers about installing and using this skill.
To set up Debezium CDC for PostgreSQL, configure the connector with appropriate replication slots, publications, and schema history for DDL propagation. You need Kafka Connect deployed, then use snapshot.mode initial to capture existing data before streaming ongoing row changes.
The outbox pattern in Debezium ensures transactional CDC by writing change events to a dedicated outbox table within the same database transaction. Use it to guarantee exactly-once delivery semantics and prevent data inconsistency between source tables and Kafka topics.
Handle delete tombstones by configuring sink connectors to process null-value Kafka messages as deletions, and manage schema evolution using schema history topics for DDL propagation. Apply Single Message Transforms like ExtractNewRecordState to shape Debezium change events correctly.
Yes, Debezium CDC events can integrate with Flink streaming and Iceberg lakehouse sinks using upsert and CDC semantics. This requires applying ExtractNewRecordState flattening transforms and configuring idempotent sink connectors to handle change event parsing correctly.
Debezium CDC supports MySQL and Oracle replication by configuring binlog settings and schema history for DDL propagation. Each database requires specific connector prerequisites and snapshot modes to ensure correct change event capture and structured export to Kafka topics.
anton-abyzovConfigure and manage Kafka Connect source and sink connectors.
ivanshamaevBuild StarRocks CDC pipelines from PostgreSQL, MySQL, or Oracle with Debezium, Kafka, or Flink.
delorenjDesign event-driven architectures with Kafka, RabbitMQ, and SQS/SNS.
sandeepmvlConfigure Debezium CDC to stream Rails PostgreSQL changes to Kafka.
