starrocks-concurrency-optimizer

Tune StarRocks parallelism and resource groups for high-concurrency BI workloads.

14|1|Updated May 5, 2026
One-click install
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-concurrency-optimizer
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: starrocks-concurrency-optimizer
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/starrocks_group_skills/starrocks_concurrency_optimizer
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-concurrency-optimizer

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves StarRocks performance instability in high-concurrency BI workloads, where p99 latency spikes and CPU saturation happen due to excessive parallelism, connection bottlenecks, and missing isolation controls.

Core Features & Use Cases

  • Pipeline DOP tuning: Reduce or cap pipeline_dop globally, per-session, or per-query to prevent thread oversubscription during bursts of small dashboard queries.
  • Workload isolation with resource groups: Create and size short_query resource groups to protect interactive BI traffic from ETL/ad-hoc contention using CPU and concurrency limits.
  • Queueing and connection handling: Enable StarRocks query queueing and properly size/route FE connections (including pool sizing and load balancing across FE nodes) to avoid thundering-herd overload.
  • Scan and point-lookup optimization: Increase scan parallelism via tablet/bucket distribution for concurrency, and enable Primary Key table direct lookup with persistent index for sub-millisecond key-value access.
  • Cache for repeated dashboard patterns: Enable query cache and explain cache eligibility/invalidation so repeated historical dashboard queries return faster.

Quick Start

Apply this skill to StarRocks by setting a lower global pipeline DOP, creating a short_query resource group for BI users, enabling the appropriate query queue settings, and turning on query cache and point-query detail cache for your dashboard/lookup patterns.

Frequently Asked Questions about starrocks-concurrency-optimizer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
Why does StarRocks p99 latency spike under high BI dashboard concurrency?

StarRocks p99 latency spikes under high BI dashboard concurrency occur due to excessive execution parallelism causing CPU thread oversubscription, missing resource group isolation for ETL contention, and FE connection pool exhaustion.

How do I tune pipeline_dop in StarRocks to prevent thread oversubscription?

To prevent StarRocks thread oversubscription, reduce or cap pipeline_dop globally, per-session, or per-query. This limits execution parallelism during bursts of concurrent small dashboard queries to stabilize CPU saturation.

How to isolate interactive BI queries from ETL workloads in StarRocks?

Isolate interactive BI queries from StarRocks ETL workloads by creating and sizing a short_query resource group. This protects dashboard traffic using strict CPU and concurrency limits to prevent resource contention.

Can I use query queue and query cache to handle 100+ concurrent StarRocks queries?

Yes, you can handle 100+ concurrent StarRocks queries by enabling the query queue to manage thundering-herd overload and turning on query cache to return repeated historical dashboard query patterns faster.

What is the best way to optimize StarRocks Primary Key point-lookups for submillisecond access?

Optimize StarRocks Primary Key point-lookups for submillisecond access by enabling direct lookup with persistent index and point-query detail cache, bypassing full table scans for key-value access.

When should I adjust StarRocks scan parallelism for concurrent workloads?

Adjust StarRocks scan parallelism for concurrent workloads when tablet or bucket distribution is insufficient for the query volume. Increasing scan parallelism helps handle bursty dashboard queries by distributing the read load.