skills.rest
ivanshamaev
ivanshamaevCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

starrocks-cdc-pipeline

Build StarRocks CDC pipelines from PostgreSQL, MySQL, or Oracle with Debezium, Kafka, or Flink.

14|1|Updated May 5, 2026
Search Tags:#kafka#cdc#debezium#flink#starrocks#exactly-once#routine load
One-click install
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-cdc-pipeline
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: starrocks-cdc-pipeline
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/starrocks_group_skills/starrocks_cdc_pipeline
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-cdc-pipeline

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the challenge of continuously replicating INSERT/UPDATE/DELETE changes from transactional sources into StarRocks with correct upsert semantics and production-grade reliability.

Core Features & Use Cases

  • CDC ingestion patterns: Debezium + Kafka into StarRocks via Routine Load, or Flink CDC into StarRocks with exactly-once and delete support.
  • Mutable StarRocks landing with primary keys: Uses StarRocks PK tables with persistent index to merge changes efficiently.
  • Operational hardening: Handles schema evolution propagation, builds multi-table CDC fan-out, routes failed events to a DLQ, and provides CDC lag monitoring guidance.
  • Use Case: Stream real-time operational analytics for an e-commerce system by replicating order and customer changes from PostgreSQL/MySQL into StarRocks for near-instant dashboards.

Quick Start

Use the starrocks-cdc-pipeline skill to design a Debezium/Kafka (Routine Load) or Flink CDC (exactly-once) pipeline that applies CDC events to a StarRocks primary-key table with upsert behavior and delete handling as required.

Frequently Asked Questions about starrocks-cdc-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I replicate transactional changes into StarRocks primary-key tables?▼

Replicate transactional changes into StarRocks by applying CDC events from PostgreSQL, MySQL, or Oracle to primary-key tables for correct upsert and full DML operations. Use Debezium with Kafka via Routine Load or Flink CDC with the StarRocks sink.

How does StarRocks CDC handle DELETE operations during replication?▼

StarRocks CDC handles DELETE operations by selecting the correct routing pattern for delete semantics within your pipeline. Proper routing ensures that delete events from Debezium or Flink CDC are accurately applied to primary-key tables.

Can I use Flink CDC for exactly-once replication into StarRocks?▼

Yes, you can use Flink CDC for exactly-once replication into StarRocks by enabling Flink checkpointing with the StarRocks sink. This ensures reliable transactional change application without data duplication.

What is the best way to monitor CDC lag and handle failed events in StarRocks?▼

The best way to monitor CDC lag and handle failed events is to route failed events to a Dead Letter Queue (DLQ) and implement CDC lag monitoring. This production hardening ensures pipeline reliability and operational safety.

Does StarRocks CDC support schema evolution and multi-table fan-out?▼

Yes, StarRocks CDC supports schema evolution propagation and multi-table CDC fan-out. These features allow you to stream changes from multiple source tables and apply transformations or enrichments before landing the data.

Why do I need a persistent primary-key index for StarRocks upsert operations?▼

You need a persistent primary-key index for StarRocks upsert operations to efficiently merge INSERT, UPDATE, and DELETE changes. This indexing ensures mutable StarRocks landing tables process transactional updates correctly.

Related Skills

Explore More Software Engineering Skills→
ivanshamaevivanshamaev
14

cdc-debezium

Set up Debezium CDC pipelines exporting database row changes to Kafka.

Community
Advanced
jamescha-earleyjamescha-earley

snowflake-postgres

Sync Postgres tables as Iceberg lake tables for Snowflake analytics.

Community
Advanced
AbsolutelySkilledAbsolutelySkilled
207

real-time-streaming

Guide building and operating Kafka and Flink streaming pipelines with exactly-once semantics.

Official
Advanced
ivanshamaevivanshamaev
14

apache-flink

Design Apache Flink streaming pipelines with event-time windowed aggregations and exactly-once semantics.

Community
Advanced