skills.rest
ricardolui
ricardoluiCOMMUNITY
Explore all skills by this author →
GitHub

developing-kafka-java-client

Generates Java Kafka producer and consumer projects with Schema Registry for Confluent Cloud, Docker, or WarpStream.

Updated Jul 7, 2026
Search Tags:#java#kafka#avro#confluent cloud#producer consumer#schema registry#warpstream
One-click install
npx skills add https://github.com/ricardolui/gcp-custom-agent-skills --skill developing-kafka-java-client-ricardolui
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: developing-kafka-java-client
Source: https://github.com/ricardolui/gcp-custom-agent-skills/tree/main/developing-kafka-java-client
Command: npx skills add https://github.com/ricardolui/gcp-custom-agent-skills --skill developing-kafka-java-client-ricardolui

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Setting up a Java Kafka client involves many error-prone decisions: serializer configuration, Schema Registry integration, environment-specific connection settings, and correct producer/consumer lifecycle patterns. This Skill scaffolds or integrates production-grade Java Kafka client code following Confluent best practices, avoiding common mistakes like silent schema auto-registration or per-message producer creation. ## Core Features & Use Cases - Project Scaffolding or In-Place Integration: Generates a complete Maven or Gradle project (producer, consumer, or both) from scratch, or modifies an existing Spring Boot/Quarkus application to add Schema Registry serializers without discarding existing code. - Multi-Environment Support: Targets Confluent Cloud (SASL_SSL), local Docker (PLAINTEXT with docker-compose.yml), or WarpStream with throughput-optimized client overrides like disabled idempotence and zone-aware routing. - Schema Management: Generates Avro (default), JSON Schema, or Protobuf schemas with explicit registration via CachedSchemaRegistryClient and auto.register.schemas=false, including multi-event union schemas for topics carrying multiple event types. - Use Case: A developer needs a Java producer and consumer for IoT sensor readings on Confluent Cloud using Gradle. The Skill asks clarifying questions, confirms the setup, then generates AvroProducer, AvroConsumer, KafkaConfig, the Avro schema, build file, and passing unit tests. ## Quick Start Ask the assistant to create a Java Kafka producer and consumer for your data fields on Confluent Cloud, local Docker, or WarpStream, specifying Maven or Gradle and your preferred send pattern.

Frequently Asked Questions about developing-kafka-java-client

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a Java Kafka producer with Schema Registry?▼

Use the KafkaProducer with a Confluent serializer such as KafkaAvroSerializer, and register the schema explicitly via CachedSchemaRegistryClient with auto.register.schemas=false. Create one producer instance in main() and reuse it, since producers are thread-safe and expensive to construct.

Should I use the Kafka Consumer API or the Share Consumer API?▼

Use the regular Consumer API (KafkaConsumer) by default: each partition maps to one consumer and per-key ordering is preserved. Choose the Share Consumer API (KafkaShareConsumer) for queue-like semantics where multiple consumers process the same partition and you scale beyond the partition count.

Does the Java Kafka client work with WarpStream?▼

Yes, WarpStream is Kafka-protocol-compatible, but client defaults should change: disable idempotence, raise max.in.flight.requests.per.connection and batch.size, set large fetch sizes, and append ws_az=<az> to client.id for zone-aware routing. Do not set fetch.min.bytes, which WarpStream does not support.

Can I add Schema Registry to an existing Spring Boot Kafka app?▼

Yes. Replace the raw StringSerializer with a Confluent serializer like KafkaAvroSerializer, generate a schema from your existing message structure, and add the Confluent Maven repository and serializer dependency to your existing build file. The existing code is modified in place rather than replaced.

Why is my Kafka consumer rebalancing slowly on Kafka 4.x?▼

Set group.protocol=consumer to enable the next-generation rebalance protocol (KIP-848), which moves assignment computation to the broker and reconciles incrementally. This eliminates the stop-the-world barrier of the classic protocol and requires Kafka 4.0+ on both client and broker.

Related Skills

Explore More Software Engineering Skills→
confluentincconfluentinc
42

developing-kafka-python-client

Create a Python Kafka client project with Schema Registry and JSON Schema validation.

Official
Intermediate
heaptracetechnologyheaptracetechnology
13

contract-test

Automate API contract testing with schema validation and OpenAPI checks.

Community
Advanced
Samuelca6399Samuelca6399

real-time-streaming

Design and operate Kafka and Flink pipelines with exactly-once processing.

Community
Advanced
SovranBitcoinSovranBitcoin
23

expo-dev-client

Build and distribute Expo development clients for testing native code on devices.

Official
Intermediate