kafka-expert

Explain Apache Kafka architecture, producers, consumers, and cluster operations.

Updated Feb 15, 2026

Search Tags:#distributed systems #message queue #kafka #data pipelines #stream processing #event streaming #apache kafka

Ready to run

npx skills add https://github.com/Wbunker/skills-repo --skill kafka-expert-wbunker

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: kafka-expert
Source: https://github.com/Wbunker/skills-repo/tree/main/kafka-expert
Command: npx skills add https://github.com/Wbunker/skills-repo --skill kafka-expert-wbunker

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill provides comprehensive guidance on Apache Kafka, enabling users to design robust architectures, configure producers and consumers effectively, troubleshoot delivery issues, and manage Kafka clusters securely and efficiently.

Core Features & Use Cases

Kafka Architecture Design: Understand brokers, topics, partitions, and KRaft.
Producer/Consumer Configuration: Optimize settings for reliability and performance.
Debugging & Operations: Troubleshoot delivery issues, manage clusters, and monitor health.
Advanced Topics: Kafka Connect, Kafka Streams, security, and cross-cluster replication.
Use Case: Design a highly available Kafka cluster for a new microservices application, ensuring reliable data flow between services.

Quick Start

Explain the Kafka producer configuration for exactly-once delivery.

Frequently Asked Questions about kafka-expert

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I configure a Kafka producer for exactly-once delivery?▼

Kafka producer configuration for exactly-once delivery requires enabling idempotent producers and utilizing transactional APIs to guarantee message delivery without duplication. This ensures reliable data flow across distributed system components.

What is KRaft mode in Apache Kafka architecture?▼

KRaft mode in Apache Kafka architecture is a consensus protocol that replaces ZooKeeper for cluster metadata management. It simplifies operations by embedding metadata directly into brokers, improving scalability and stream processing reliability.

How do I troubleshoot Kafka consumer message delivery issues?▼

Troubleshooting Kafka consumer message delivery issues involves examining consumer group offsets, partition rebalancing events, and commit configurations. Debugging these mechanisms identifies data pipeline bottlenecks and ensures reliable event streaming.

Does Kafka Connect work with stream processing for data pipelines?▼

Yes, Kafka Connect integrates with stream processing to build robust data pipelines by importing and exporting data between Kafka and external systems. This simplifies reliable data flow management without requiring custom producer or consumer code.

What is the best way to monitor Kafka cluster health?▼

The best way to monitor Kafka cluster health involves tracking broker metrics, partition lag, and network throughput. Effective cluster operations monitoring ensures high availability for event streaming and prevents data pipeline failures.

When do I need cross-cluster replication in Kafka?▼

Cross-cluster replication in Kafka is needed to ensure disaster recovery and high availability across geographically distributed systems. This architecture mirrors data pipelines to secondary clusters, preventing message queue downtime during regional failures.