What problem does it solve?
This Skill solves the problem of delivering sub-minute analytics from Kafka streams while keeping BI queries low-latency and dashboard-friendly in StarRocks.
Core Features & Use Cases
- Kafka → Routine Load ingestion: set up near-real-time ingestion with tuned batch interval, concurrency, JSON mapping, strict mode, and UTC handling.
- Primary Key real-time tables: design StarRocks Primary Key tables with range partitioning and low-latency storage settings for upsert-like updates.
- Real-time serving for dashboards: write time-window metric queries and use async materialized views for repeated aggregations to reduce query cost.
- Low-latency multi-table querying: apply colocate join patterns so join execution avoids unnecessary shuffle during real-time queries.
- Operational stability: isolate ingestion vs OLAP workload using resource groups to prevent contention.
- Freshness monitoring: check ingestion lag (seconds behind latest record) to drive alerts for real-time dashboards.
Use Case example: you ingest order events from Kafka and power a live “revenue last 15 minutes” dashboard with pre-aggregated MV-backed queries refreshed every minute.
Quick Start
Create a StarRocks Primary Key table for your events, configure Routine Load from your Kafka topic, and query the last-15-minute metrics in Grafana using minute-level time aggregation.
