skills.rest
ivanshamaev
ivanshamaevCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

starrocks-medallion-architecture

Design StarRocks Bronze, Silver, and Gold medallion schemas for CDC pipelines.

14|1|Updated May 5, 2026
Search Tags:#deduplication#medallion architecture#cdc#upsert#starrocks#materialized view#insert overwrite
One-click install
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-medallion-architecture
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: starrocks-medallion-architecture
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/starrocks_group_skills/starrocks_medallion_architecture
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-medallion-architecture

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It helps you design a reliable StarRocks data platform where raw CDC ingestion stays immutable in Bronze, deduplicated business state is maintained in Silver, and analytics-ready aggregates are produced in Gold.

Core Features & Use Cases

  • Bronze raw landing for CDC: ingest event streams into Duplicate Key tables to preserve all versions of records.
  • Silver cleansed upsert + dedup: transform and deduplicate Bronze into Primary Key tables using latest-op-per-key logic and explicit delete propagation.
  • Gold analytics aggregates: build daily business metrics with Aggregate Key tables or Materialized Views, using INSERT OVERWRITE patterns for re-runs.
  • Incremental pipeline patterns: implement partitioned loads using watermarks/time windows and orchestrate Bronze → Silver → Gold flows (e.g., Airflow).

Quick Start

Use the starrocks-medallion-architecture Skill to generate a Bronze→Silver→Gold StarRocks schema and incremental loading SQL for your CDC stream, including deduplication, upserts, delete handling, and Gold daily aggregation.

Frequently Asked Questions about starrocks-medallion-architecture

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design a Bronze Silver Gold medallion architecture in StarRocks for CDC ingestion?▼

Designing a medallion architecture in StarRocks for CDC involves using Duplicate Key tables for Bronze raw landing, Primary Key tables for Silver deduplication and upserts, and Aggregate Key tables or Materialized Views for Gold analytics.

How does StarRocks handle CDC upserts and delete propagation in a Silver layer?▼

StarRocks handles CDC upserts in the Silver layer using Primary Key tables to apply latest-row deduplication logic and explicit delete propagation, ensuring the cleansed business state remains accurate and current.

What is the best way to build incremental data pipelines in StarRocks for daily analytics aggregation?▼

The best way to build incremental pipelines in StarRocks is using partitioned loads with watermarks or time windows, orchestrating Bronze to Silver to Gold flows, and applying INSERT OVERWRITE patterns for re-runnable Gold daily aggregation.

When should I use Materialized Views vs Aggregate Key tables for Gold layer metrics in StarRocks?▼

In StarRocks Gold layers, both Materialized Views and Aggregate Key tables build daily business metrics. Use partitioned INSERT OVERWRITE patterns with either approach to enable idempotent re-runs and incremental refresh of analytics-ready aggregates.

Can I use StarRocks Duplicate Key tables to preserve all raw CDC event versions in a Bronze layer?▼

Yes, you can use StarRocks Duplicate Key tables for Bronze raw CDC landing to preserve all versions of records immutably, ensuring no source event data is lost before transformation and deduplication in downstream Silver layers.

Related Skills

Explore More Software Engineering Skills→
ivanshamaevivanshamaev
14

starrocks-realtime-modeling

Designs StarRocks ingestion and modeling for mutable tables via Kafka, Flink, or CDC streams.

Community
Advanced
prashsubprashsub
1

lakehouse-monitoring-comprehensive

Set up Databricks Lakehouse Monitoring for Gold layer tables with custom metrics.

Community
Advanced
ivanshamaevivanshamaev
14

medallion_architecture

Design Bronze, Silver, and Gold Iceberg lakehouse layers with MERGE-based deduplication.

Community
Advanced
elihuvillarauselihuvillaraus
2

data-eng

Design and build ETL/ELT pipelines with Medallion Architecture on cloud platforms.

Community
Advanced