skills.rest
tomz
tomzCOMMUNITY
Explore all skills by this author →
Website𝕏@tomzengGitHub

azure-hdinsight-migration-kafka-to-fabric-rti

Migrate HDInsight Kafka workloads to Fabric RTI and Azure Event Hubs Kafka endpoints.

Updated Apr 27, 2026
Search Tags:#kafka#kql#eventhouse#hdinsight#fabric-rti#eventstreams#mirrormaker
One-click install
npx skills add https://github.com/tomz/agent-skills --skill azure-hdinsight-migration-kafka-to-fabric-rti
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: azure-hdinsight-migration-kafka-to-fabric-rti
Source: https://github.com/tomz/agent-skills/tree/main/azure-hdinsight-migration-kafka-to-fabric-rti
Command: npx skills add https://github.com/tomz/agent-skills --skill azure-hdinsight-migration-kafka-to-fabric-rti

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

End-to-end migration playbook for moving Apache Kafka workloads from Azure HDInsight Kafka clusters to Microsoft Fabric Real-Time Intelligence (Eventstreams, Eventhouse/KQL DB, Activator, Real-Time Dashboards) and/or Azure Event Hubs Kafka endpoint — assessment, topic + consumer-group inventory, schema/serde migration, MirrorMaker 2 cutover, producer/consumer code rewrites, security mapping (ESP/Kerberos → Entra ID + Managed Identity), KQL ingestion mappings, validation.

Core Features & Use Cases

  • Assessment: inventory HDInsight topics, consumer groups, brokers; map to EH topics and Fabric destinations.
  • Cutover planning: guidance for MirrorMaker 2 cutover, offset translation, safety checks.
  • Code & security migration: update producer/consumer configs; migrate from ESP/Kerberos to Entra ID OAuth or SASL.
  • Ingestion & analytics: define KQL mappings, Eventstream wiring, and dashboards.

Quick Start

Run this playbook to assess your HDInsight Kafka deployment and begin the MM2 cutover to Fabric RTI.

Frequently Asked Questions about azure-hdinsight-migration-kafka-to-fabric-rti

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I migrate Kafka workloads from HDInsight to Fabric Real-Time Intelligence?▼

Migrate HDInsight Kafka workloads to Fabric RTI by assessing topics and consumer groups, executing a MirrorMaker 2 cutover, updating producer/consumer code, and defining KQL ingestion mappings for Eventhouse.

What's the best way to handle security when moving Kafka from ESP/Kerberos to Fabric RTI?▼

Security migration from ESP/Kerberos to Fabric RTI involves mapping authentication to Entra ID OAuth or SASL and configuring Managed Identity for producer and consumer clients.

How does MirrorMaker 2 cutover work for HDInsight Kafka to Event Hubs migration?▼

MirrorMaker 2 cutover replicates topics from HDInsight Kafka to Azure Event Hubs Kafka endpoint, requiring offset translation and safety checks to ensure no message loss during the transition.

Can I use KQL ingestion mappings to route Kafka events into Eventhouse after migration?▼

Yes, KQL ingestion mappings define how Kafka events are structured and ingested into the Eventhouse KQL database, enabling real-time analytics and dashboarding in Fabric RTI.

Do I need to rewrite producer and consumer code when migrating from HDInsight Kafka to Event Hubs?▼

Producer and consumer code rewrites are required to update client configurations, ensuring compatibility with the Azure Event Hubs Kafka endpoint and new Entra ID authentication.

What are the limitations of using MirrorMaker 2 for HDInsight Kafka migration?▼

MirrorMaker 2 migration requires careful offset translation and cutover sequencing; schema and serde compatibility must be validated to prevent ingestion errors in Fabric Eventstreams.

Related Skills

Explore More Software Engineering Skills→
clawdataclawdata
24

kafka

Manage Kafka clusters, topics, and messages with CLI tools.

Official
Intermediate
fgarofalo56fgarofalo56
6

fabric-rti-mcp

Query Eventhouse with KQL and build Eventstreams for real-time data processing.

Community
Advanced
delorenjdelorenj
1

bloodbank-event-publisher

Publish and consume typed events on the Bloodbank event bus with RabbitMQ routing.

Community
Advanced
manutejmanutej
61

aws-cloud-services

Plan AWS cloud deployments with structured guidance for S3, Lambda, DynamoDB, and CloudFormation.

Community
Advanced