redpanda

What problem does it solve?

This Skill helps you deploy and operate Redpanda as a Kafka-compatible streaming platform while configuring topics, clients, security, and tiered storage for reliable data engineering.

Core Features & Use Cases

• Cluster deployment (Docker/Kubernetes/bare metal): Stand up single-node for development and multi-node clusters for staging/production.
• Topic and consumer/group management with rpk: Create topics with retention/compaction settings, inspect leaders/replicas, manage consumer offsets and lag.
• Performance and reliability tuning: Configure producer durability/throughput and consumer fetch/commit/rebalance behavior.
• Tiered storage (Shadow Indexing): Offload historical segments to S3/GCS/ABS and control remote read/write/delete semantics.
• Schema Registry and Kafka Connect compatibility: Use the built-in Schema Registry APIs and run standard Kafka Connect workers against Redpanda.
• Monitoring and security: Enable SASL/SCRAM, TLS/mTLS, ACLs, and instrument metrics via Prometheus/Grafana.

Quick Start

Configure a 3-broker Redpanda cluster with Schema Registry and tiered storage, then create a topic using rpk with the appropriate retention and remote read/write settings.