skills.rest
roanbrasil
roanbrasilCOMMUNITY
Explore all skills by this author →
Website𝕏@roanbrasilGitHub

kafka-mastery

Configure and troubleshoot Apache Kafka for event-driven systems.

5|1|Updated Jun 17, 2026
Search Tags:#event streaming#apache kafka#schema registry#kafka streams#kafka connect#exactly-once semantics#consumer lag
One-click install
npx skills add https://github.com/roanbrasil/engineer-grade-agent-skills --skill kafka-mastery
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: kafka-mastery
Source: https://github.com/roanbrasil/engineer-grade-agent-skills/tree/main/skills/kafka-mastery
Command: npx skills add https://github.com/roanbrasil/engineer-grade-agent-skills --skill kafka-mastery

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill eliminates the common production headaches of misconfigured Kafka clusters, unplanned consumer rebalances, data loss from incorrect offset handling, and fragile stream processing topologies that plague event-driven systems.

Core Features & Use Cases

  • Production Configuration Templates: Provides pre-tuned producer, consumer, and broker configs for throughput, latency, and durability tradeoffs, with clear guidance on when to use each profile.
  • Operational Troubleshooting: Covers consumer lag monitoring, rebalance loop root causes, message size limits, and partition count best practices to resolve common production incidents.
  • Event Streaming Implementation: Supports Kafka Streams topology design, Schema Registry compatibility rules, and Kafka Connect setup for real-time data pipelines.
  • Use Case: For a team building an order processing pipeline, this skill helps configure idempotent producers for exactly-once semantics, switch to CooperativeStickyAssignor to eliminate rebalance downtime, and design windowed stream topologies for per-minute sales aggregations.

Quick Start

Use the kafka-mastery skill to design a fault-tolerant Kafka Streams topology for real-time order enrichment with windowed aggregations and Schema Registry compatibility.

Frequently Asked Questions about kafka-mastery

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fix consumer lag in Apache Kafka deployments?▼

Fix Apache Kafka consumer lag by monitoring partition distribution, optimizing consumer fetch configurations, and applying CooperativeStickyAssignor to eliminate rebalance downtime and maintain throughput.

How do I implement exactly-once semantics in Kafka event streaming?▼

Implement exactly-once semantics in Kafka by configuring idempotent producers and transactional APIs, ensuring data integrity and preventing duplicate message processing during production-grade event streaming.

What is the best way to design a Kafka Streams topology for real-time aggregations?▼

Design Kafka Streams topologies for real-time aggregations by structuring windowed operations for per-minute processing, integrating Schema Registry compatibility, and ensuring fault-tolerant stream processing.

Why does my Kafka consumer rebalance loop keep causing downtime?▼

Kafka consumer rebalance loops cause downtime due to misconfigured partition assignment strategies; switching to CooperativeStickyAssignor optimizes rebalancing and resolves unplanned production incidents.

Can I use Schema Registry to manage compatibility across Kafka Connect pipelines?▼

Use Schema Registry with Kafka Connect pipelines to enforce compatibility rules, manage evolving data schemas, and maintain reliable real-time data integration without breaking downstream consumers.

When do I need to tune producer and broker configs for Kafka throughput versus durability?▼

Tune Kafka producer and broker configs for throughput versus durability when optimizing event-driven systems, applying pre-tuned configuration templates to balance latency and data persistence.

Related Skills

Explore More Software Engineering Skills→
hung-phanhung-phan

kafka-patterns

Analyze Apache Kafka configurations and designs for durability and ordered delivery.

Community
Advanced
ivanshamaevivanshamaev
14

apache-kafka

Configure Apache Kafka topics, producers, consumers, connectors, and monitoring.

Community
Advanced
belokonmbelokonm
8

kafka-engineer

Design Kafka architectures and configure Connect pipelines for real-time data streaming.

Community
Advanced
manutejmanutej
61

kafka-stream-processing

Implement Apache Kafka producers, consumers, streams, and connectors for real-time data processing.

Community
Advanced