starrocks-stream-load

What problem does it solve?

It solves the challenge of reliably ingesting small to micro-batch CSV or JSON datasets into StarRocks without building a heavy ingestion pipeline.

Core Features & Use Cases

• Push-based ingestion via HTTP Stream Load: Send CSV or JSON payloads to StarRocks using an HTTP PUT workflow.
• Flexible parsing and column mapping: Configure delimiters and JSON jsonpaths, including optional stripping of an outer JSON array.
• Upsert/partial update semantics for Primary Key tables: Update only specified columns using partial_update (and optional merge commit mode for high concurrency).
• Operational safety knobs: Use label-based idempotency, filtering controls, strict_mode validation, timeouts, and error diagnostics through ErrorURL.
• Use case: A Python ETL step produces an in-memory batch of events and needs to push them into a StarRocks Primary Key table with upsert behavior and retry safety.

Quick Start

Use the starrocks-stream-load skill to send your CSV or NDJSON records to StarRocks via the _stream_load endpoint, providing a unique label and explicit column mapping for a successful load.