skills.rest
ricardolui
ricardoluiCOMMUNITY
Explore all skills by this author →
GitHub

kafka-schema-registry

Scan projects for Kafka usage, extract schemas, tag PII fields, and generate Terraform for Confluent Schema Registry.

Updated Jul 7, 2026
Search Tags:#terraform#kafka#confluent#avro#schema extraction#schema registry#pii tagging
One-click install
npx skills add https://github.com/ricardolui/gcp-custom-agent-skills --skill kafka-schema-registry-ricardolui
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: kafka-schema-registry
Source: https://github.com/ricardolui/gcp-custom-agent-skills/tree/main/kafka-schema-registry
Command: npx skills add https://github.com/ricardolui/gcp-custom-agent-skills --skill kafka-schema-registry-ricardolui

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Teams adopting Kafka often lack visibility into which applications produce or consume events, whether schemas are registered, and whether risky settings like auto.register.schemas are enabled. This Skill audits a repository for Kafka usage and produces schemas, Terraform configs, and a migration report so Schema Registry adoption becomes a governed, repeatable process. ## Core Features & Use Cases - Kafka Application Detection: Scans build files and source code across Java, Python, .NET, Go, and Node.js to catalog producers, consumers, topics, and serializers. - Schema Extraction & PII Tagging: Infers Avro, JSON Schema, or Protobuf schemas from data models and inline data, tagging fields like email, ssn, and credit card with confluent:tags. - Terraform Generation: Produces Confluent provider configs (schemas.tf, tags.tf, flagged-auto-register.tf) with per-category rollout ordering and a full schema-report.md. - Use Case: Point it at a monorepo before a compliance audit to discover all Kafka producers, flag auto-registration risks, tag PII fields for GDPR, and generate Terraform to register every schema under infrastructure-as-code control. ## Quick Start Ask the assistant to scan your project folder for Kafka usage, extract schemas, tag PII fields, and generate Terraform for Schema Registry registration.

Frequently Asked Questions about kafka-schema-registry

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract Kafka schemas from an existing project?▼

Scan the repository for build files and producer patterns, then infer schemas from data models such as Java classes, Pydantic models, Go structs, or TypeScript interfaces. The extracted schemas are written as Avro, JSON Schema, or Protobuf files organized by topic.

How do I generate Terraform for Confluent Schema Registry?▼

The skill generates a terraform/ directory with providers.tf, variables.tf, schemas.tf, and outputs.tf using the Confluent provider. Each confluent_schema resource references an extracted schema file, and schemas from auto-registering producers are placed commented-out in flagged-auto-register.tf.

Why is auto.register.schemas=true a risk in Kafka producers?▼

Auto-registration allows uncontrolled schema evolution, meaning breaking changes can be registered at runtime without review and break downstream consumers. The recommended fix is to register schemas via Terraform, set auto.register.schemas=false, and enable use.latest.version.

Does this support languages besides Java for Kafka schema extraction?▼

Yes, it detects Kafka usage in Java, Python, .NET, Go, and Node.js/TypeScript projects. It recognizes frameworks like spring-kafka, confluent-kafka, kafka-python, sarama, kafkajs, and Confluent.Kafka, and infers schemas from each language's data model conventions.

How are PII fields tagged in Kafka schemas?▼

Field names are matched against patterns like email, ssn, phone, address, and credit_card, then annotated with confluent:tags such as PII, PRIVATE, SENSITIVE, or PHI. A tags.tf file creates the corresponding confluent_tag resources so schemas can reference them.

What happens when multiple event types share one Kafka topic?▼

The skill detects multi-schema topics and generates a wrapper schema using oneOf for JSON Schema, a union for Avro, or oneof for Protobuf. Terraform is generated with schema_reference blocks pointing to the individual event schemas.

Related Skills

Explore More Software Engineering Skills→
indicate-euindicate-eu
7

build-catalog

Regenerate static catalog data files from source JSON for GitHub Pages.

Official
Basic
confluentincconfluentinc
42

developing-kafka-python-client

Create a Python Kafka client project with Schema Registry and JSON Schema validation.

Official
Intermediate
proffesor-for-testingproffesor-for-testing
436

contract-testing

Verify microservice API providers against consumer-driven Pact contracts in CI/CD pipelines.

Community
Advanced
medmomedmo

avro-schema-generator

Generate compliant Avro schemas for Kafka events from EDA design documents.

Community
Advanced