skills.rest
ivanshamaev
ivanshamaevCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

starrocks-broker-load

Automate bulk data loading into StarRocks via Broker Load from cloud storage.

14|1|Updated May 5, 2026
Search Tags:#starrocks#airflow integration#broker load#bulk etl#data lake ingestion#s3 parquet#load monitoring
One-click install
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-broker-load
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: starrocks-broker-load
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/starrocks_group_skills/starrocks_broker_load
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill starrocks-broker-load

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the problem of bulk-loading large datasets from object storage or HDFS into StarRocks with repeatable, production-friendly control over formats, authentication, and load monitoring.

Core Features & Use Cases

  • Bulk Broker Load into StarRocks: Perform LOAD LABEL from S3/HDFS/GCS/Azure/MinIO sources for large batch ETL without tying up the caller synchronously.
  • Multi-format ingestion: Load CSV, Parquet, and ORC with schema mapping expressions and format-specific configuration.
  • Operational safety & observability: Track progress via SHOW LOAD, cancel jobs with CANCEL LOAD, and tune parallelism and timeouts for predictable throughput.
  • Use Case: Daily/hourly landing-zone batch ETL where partitioned lake files (e.g., dt=YYYY-MM-DD/*.parquet) must be ingested into one or multiple StarRocks tables and then followed by ANALYZE for fresh optimizer stats.

Quick Start

Use the starrocks-broker-load Skill to load Parquet partition files from S3 into a target table by providing the correct LOAD LABEL statement, broker authentication, and PROPERTIES such as timeout for an async batch run.

Frequently Asked Questions about starrocks-broker-load

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I bulk load Parquet files from S3 into StarRocks?▼

To bulk load Parquet files from S3 into StarRocks, use Broker Load with a defined `LOAD LABEL` and `WITH BROKER` credentials. This asynchronous method handles large batch ETL ingestion without tying up the caller synchronously.

What is the best way to monitor and cancel a StarRocks Broker Load job?▼

The best way to monitor a StarRocks Broker Load job is using `SHOW LOAD` to track progress. You can cancel running jobs with `CANCEL LOAD` and tune parallelism or timeouts via `PROPERTIES` for predictable throughput.

Can I load CSV and ORC files from HDFS into StarRocks asynchronously?▼

Yes, you can load CSV and ORC files from HDFS into StarRocks asynchronously. Broker Load supports multi-format ingestion from HDFS, S3, GCS, MinIO, and Azure with schema mapping expressions and format-specific configuration.

Does StarRocks Broker Load support multi-table atomic loads for daily partition ingestion?▼

Yes, StarRocks Broker Load supports multi-table atomic loads for daily partition ingestion. It applies `LOAD LABEL` syntax and optional column mapping expressions to ingest partitioned lake files reliably into one or multiple tables.

How do I tune StarRocks Broker Load properties for large file backfills?▼

To tune StarRocks Broker Load for large file backfills, adjust `PROPERTIES` settings like `timeout` to ensure reliable execution. This provides explicit operational control over asynchronous batch ETL workflows.

Why does my Broker Load job require explicit operational control in StarRocks?▼

Broker Load jobs require explicit operational control in StarRocks because they execute asynchronously for large batch ETL. Using `SHOW LOAD`, `CANCEL LOAD`, and `PROPERTIES` tuning ensures reliable throughput and prevents hanging jobs.

Related Skills

Explore More Data & Analytics Skills→
ivanshamaevivanshamaev
14

airflow-starrocks-etl-best-practices

Design idempotent Airflow ETL pipelines loading data into StarRocks.

Community
Advanced
ivanshamaevivanshamaev
14

airflow-starrocks-pipeline

Orchestrate StarRocks ingestion from Airflow with Broker Load polling and Stream Load.

Community
Advanced
ivanshamaevivanshamaev
14

airflow-starrocks-backfill

Automate StarRocks historical backfills with Airflow using Broker Load or INSERT OVERWRITE.

Community
Advanced
StarRocksStarRocks
73

import

Diagnose StarRocks import slowness, timeouts, and RPC failures.

Official
Advanced