skills.rest
belokonm
belokonmCOMMUNITY
Explore all skills by this author →
GitHub

kafka-engineer

Design Kafka architectures and configure Connect pipelines for real-time data streaming.

8|11|Updated Feb 15, 2026
Search Tags:#real-time#kafka#kafka-connect#schema-registry#event-streaming#stream-processing#ksqldb
One-click install
npx skills add https://github.com/belokonm/claude-supercode-skills --skill kafka-engineer-belokonm
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: kafka-engineer
Source: https://github.com/belokonm/claude-supercode-skills/tree/main/kafka-engineer-skill
Command: npx skills add https://github.com/belokonm/claude-supercode-skills --skill kafka-engineer-belokonm

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Apache Kafka and event streaming architectures often become complex to design, deploy, and maintain at scale; this skill provides expert guidance for building fault-tolerant, real-time data pipelines.

Core Features & Use Cases

  • Kafka Connect pipelines (CDC, S3, JDBC) for scalable data integration.
  • Kafka Streams / ksqlDB based stream processing with exactly-once or idempotent semantics.
  • Schema Registry and security hardening (ACLs, mTLS) for production readiness.
  • Troubleshooting and performance tuning (brokers, topics, lag, retention, replication).

Quick Start

Set up a real-time Kafka pipeline by configuring a CDC source from PostgreSQL to S3 via Kafka Connect and deploying a Kafka Streams processor.

Frequently Asked Questions about kafka-engineer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up a Kafka Connect CDC pipeline from PostgreSQL to S3?▼

Set up a Kafka Connect CDC pipeline by configuring a PostgreSQL source connector and an S3 sink connector, routing change events through Kafka topics to achieve scalable real-time data integration into object storage.

When should I use ksqlDB instead of Kafka Streams for stream processing?▼

Use ksqlDB for stream processing when you prefer SQL-like syntax for building queries, and use Kafka Streams when you need to implement exactly-once or idempotent semantics within custom Java application logic.

How does Schema Registry harden Kafka security for production event streaming?▼

Schema Registry hardens Kafka security by enforcing schema compatibility rules for topic data, which pairs with security configurations like ACLs and mTLS to ensure production-ready, fault-tolerant event streaming pipelines.

What is the best way to troubleshoot Kafka consumer lag and broker performance issues?▼

Troubleshoot Kafka consumer lag and broker performance issues by analyzing topic partitions, replication factors, and retention configurations to identify bottlenecks and tune operational throughput for robust pipelines.

Can I use Kafka for event-driven microservices without managing complex broker configurations?▼

Kafka supports event-driven microservices but requires configuring topics, partitions, replication, and security settings to operate robustly, meaning you must manage these architectural components for fault-tolerant real-time data streaming.

Related Skills

Explore More Software Engineering Skills→
heyAyushhheyAyushh
2

kafka-topic-setup

Creates and configures Kafka topics for the hook event pipeline.

Community
Intermediate
ancolemanancoleman
503

streaming-data

Build event streaming pipelines with Kafka, Pulsar, Redpanda, and RabbitMQ.

Community
Advanced
anton-abyzovanton-abyzov
156

confluent-kafka-connect

Configure and manage Kafka Connect source and sink connectors.

Community
Advanced
cosmicstack-labscosmicstack-labs
364

message-queues

Design message-driven architectures with RabbitMQ, Kafka, SQS, and pub/sub patterns.

Official
Intermediate