skills.rest
yunqiqiliang
yunqiqiliangCOMMUNITY
Explore all skills by this author →
GitHub

clickzetta-cdc-sync-pipeline

Automate real-time CDC synchronization from MySQL and PostgreSQL into Lakehouse.

8|3|Updated Mar 19, 2026
Search Tags:#postgresql#real-time#mysql#data-pipeline#lakehouse#cdc#data-synchronization
One-click install
npx skills add https://github.com/yunqiqiliang/clickzetta-skills --skill clickzetta-cdc-sync-pipeline
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: clickzetta-cdc-sync-pipeline
Source: https://github.com/yunqiqiliang/clickzetta-skills/tree/main/clickzetta-cdc-sync-pipeline
Command: npx skills add https://github.com/yunqiqiliang/clickzetta-skills --skill clickzetta-cdc-sync-pipeline

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

CDC pipelines that continuously mirror or merge multi-table data from MySQL or PostgreSQL into Lakehouse, keeping analytics data fresh and ready.

Core Features & Use Cases

  • Full-database mirror: synchronize all tables and auto-adapt new ones as they appear.
  • Multi-table mirror: select specific tables and automatically detect schema changes.
  • Multi-table merge: consolidate sharded sources into a single target table with optional source metadata.
  • Use Case: If you have ecommerce orders split across schemas, this Skill merges them into a single Orders table in Lakehouse while preserving origin information.

Quick Start

Create a real-time CDC task by selecting one of the three modes, then point to your source MySQL or PostgreSQL instance and the Lakehouse target.

Frequently Asked Questions about clickzetta-cdc-sync-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I sync MySQL data to a Lakehouse in real time?▼

You can sync MySQL data to a Lakehouse in real time by using CDC to capture binlog changes and continuously mirror multi-table data. Select a sync mode, point to your source and target, and the pipeline maintains continuous data freshness.

What is the best way to consolidate sharded PostgreSQL schemas into one table?▼

The best way to consolidate sharded PostgreSQL schemas is using a multi-table merge CDC mode. It synchronizes sharded source tables into a single target Lakehouse table while preserving optional source metadata for origin tracking.

Do I need WAL access for PostgreSQL CDC data synchronization?▼

Yes, PostgreSQL CDC data synchronization requires appropriate permissions to read WALs. The pipeline needs configured source and Lakehouse data sources and an available Sync VCluster to execute real-time data replication.

Can I automatically sync new tables added to my MySQL database?▼

Yes, you can automatically sync new tables added to your MySQL database using full-database mirror mode. It synchronizes all existing tables and auto-adapts new ones as they appear, detecting schema changes without manual pipeline reconfiguration.

What are the limitations of multi-table CDC merge for sharded sources?▼

Multi-table CDC merge for sharded sources requires an available Sync VCluster and appropriate permissions to read binlogs or WALs. You must also configure both source and Lakehouse data sources before execution.

Related Skills

Explore More Data & Analytics Skills→
yunqiqiliangyunqiqiliang
8

clickzetta-realtime-sync-pipeline

Automates real-time single-table data synchronization from Kafka, MySQL, PostgreSQL, or SQL Server into Lakehouse.

Community
Advanced
xu-xiangxu-xiang
1.8k

clickhouse-io

Design ClickHouse tables and optimize queries with MergeTree family patterns.

Community
Advanced
omeraltnomeraltn

agency-data-engineer

Build ETL/ELT pipelines with bronze, silver, and gold lakehouse layers.

Community
Advanced
chicanoandres702chicanoandres702

data-engineering-data-pipeline

Design and orchestrate scalable batch and streaming data pipelines.

Community
Advanced