kafka-data-engineer

Design Kafka event-driven pipelines with topics, schemas, and processing logic.

Updated Dec 25, 2025

Search Tags:#idempotent #event-driven #kafka #schema evolution #dead-letter #topic design

One-click install

npx skills add https://github.com/hammadurrehman2006/the_evolution_of_todo --skill kafka-data-engineer

Or copy as Structured Prompt for Agent▼

Please help me install this Agent Skill.
Skill: kafka-data-engineer
Source: https://github.com/hammadurrehman2006/the_evolution_of_todo/tree/main/.claude/skills/kafka-data-engineer
Command: npx skills add https://github.com/hammadurrehman2006/the_evolution_of_todo --skill kafka-data-engineer

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps design reliable, scalable event-driven pipelines using Kafka, including topics, schemas, and processing logic for asynchronous tasks, while promoting consistent architecture across services.

Core Features & Use Cases

Topic & Schema Design: Define topics, schemas, and versioning to ensure backward compatibility in evolving systems.
Reliable Processing: Establish idempotent handlers and at-least-once delivery with clear retry and dead-letter strategies.
Observability: Monitor consumer lag, throughput, and KPI health to keep pipelines healthy.
Use Case: Model a 'task-events' topic that carries TaskEvent payloads to drive real-time dashboards and notifications.

Quick Start

Outline the 'task-events' topic, its key/schema, and a basic dead-letter workflow for failed messages.