kafka-streams-topology

Design Kafka Streams topologies with KStream, KTable, and GlobalKTable.

156|21|Updated Oct 25, 2025

Search Tags:#windowing #kafka-streams #globalktable #eos #kstream #ktable

Ready to run

npx skills add https://github.com/anton-abyzov/specweave --skill kafka-streams-topology

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: kafka-streams-topology
Source: https://github.com/anton-abyzov/specweave/tree/main/plugins/specweave-kafka-streams/skills/kafka-streams-topology
Command: npx skills add https://github.com/anton-abyzov/specweave --skill kafka-streams-topology

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides guidance on designing and implementing Kafka Streams topologies using KStream, KTable, and GlobalKTable, including common patterns, windowing, and exactly-once semantics.

Core Features & Use Cases

Core Abstractions: KStream, KTable, GlobalKTable and their use cases
Pattern & Ops: Common operations (filter, map, flatMap, branch) and joins
Windowing & EOS: Tumbling/hopping/session windows and exactly-once processing
Topology Design: Best practices for robust stream processing

Quick Start

Create a simple topology: read from a 'clicks' topic, join with a 'users' table, and write to 'enriched-clicks'.

Frequently Asked Questions about kafka-streams-topology

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I design a Kafka Streams topology to join streams and tables?▼

Design a Kafka Streams topology by defining KStream, KTable, and GlobalKTable abstractions, then apply join operations to combine data. Use KStream for event data, KTable for reference tables, and GlobalKTable for broadcast joins; specify join windows and co-partition topics as needed to enrich or correlate events.

What's the difference between KStream, KTable, and GlobalKTable in Kafka Streams?▼

KStream represents an unbounded stream of events with no state; KTable is a changelog stream representing the latest value per key, updated incrementally; GlobalKTable replicates an entire table to all instances for broadcast joins without co-partitioning. Choose based on whether you need event semantics, keyed state, or global reference lookups.

How do I implement exactly-once semantics in Kafka Streams?▼

Enable exactly-once semantics (EOS) by setting processing.guarantee to exactly_once_v2 in configuration. This ensures each record is processed and state is updated atomically, preventing duplicates even during failures. Requires compatible source and sink connectors and increases latency slightly.

What windowing strategies should I use for Kafka Streams aggregations?▼

Kafka Streams supports tumbling windows (fixed, non-overlapping intervals), hopping windows (fixed, overlapping intervals), and session windows (activity-based, close when idle). Choose tumbling for regular time-based reports, hopping for sliding analysis, and session windows for user-session analytics or burst detection.

Can I filter, map, and branch events in a Kafka Streams pipeline?▼

Yes. Use filter() to drop records matching conditions, map() to transform values, flatMap() to emit zero or more records per input, and branch() to split streams by predicates into separate subtopologies. These stateless operations compose for flexible event routing and transformation.

What are best practices for designing robust Kafka Streams topologies?▼

Co-partition topics before joins, use appropriate window retention, configure state-store cleanup policies, enable EOS for critical pipelines, handle null values explicitly, and test topology logic independently. Design for failure recovery, monitor lag and throughput, and document state-store semantics for maintainability.