starrocks-memory-tuning

What problem does it solve?

This Skill helps you diagnose and fix StarRocks memory pressure that leads to query OOM failures, BE crashes, slow performance, and unstable loads by tuning the BE memory pools, spill behavior, and FE heap settings.

Core Features & Use Cases

• Query and BE memory limit tuning: Adjusts mem_limit, per-query query_mem_limit, and resource-group mem_limit to match workload needs and prevent MemoryLimitExceeded.
• Controlled memory spill to disk: Enables and sizes spill (spill_mode, spill thresholds, spill directories, and per-query spill caps) so large joins/aggregations/sorts can finish instead of failing.
• Pool-level footprint optimization: Tunes page cache (storage_page_cache_limit), PK persistent index cache (primary_key_index_cache_capacity), and allocator behavior (jemalloc) to reduce wasted RAM and fragmentation.
• Production-grade troubleshooting workflow: Guides you through OOM log inspection, mem_tracker hierarchy review, metrics queries, and Linux OOM-killer checks to pinpoint the dominant memory consumer.
• Monitoring and alerting: Recommends Prometheus alert thresholds and dashboards based on StarRocks BE memory metrics to catch regressions early.

Example use: a BI workload intermittently fails with MemoryLimitExceeded during heavy aggregations while BE RSS climbs near the limit; you can enable spill, raise per-query caps appropriately, and reduce page cache to free headroom.

Quick Start

Diagnose the failing query by setting per-session query_mem_limit to 16 GB and enabling spill_mode='auto', then rerun the workload and confirm the dominant memory pool via BE mem_tracker and memory metrics.