azure-hdinsight-migration-spark-to-fabric

Migrate HDInsight Spark workloads to Microsoft Fabric with an end-to-end playbook.

Updated Apr 27, 2026

Search Tags:#migration #spark #fabric #lakehouse #data-migration #hdinsight #onenlake

Ready to run

npx skills add https://github.com/tomz/agent-skills --skill azure-hdinsight-migration-spark-to-fabric

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: azure-hdinsight-migration-spark-to-fabric
Source: https://github.com/tomz/agent-skills/tree/main/azure-hdinsight-migration-spark-to-fabric
Command: npx skills add https://github.com/tomz/agent-skills --skill azure-hdinsight-migration-spark-to-fabric

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

End-to-end migration playbook for moving Apache Spark workloads from Azure HDInsight to Microsoft Fabric, covering assessment, code conversion, storage migration, Hive Metastore consolidation, and cutover validation.

Core Features & Use Cases

Comprehensive assessment templates and readiness matrix for HDInsight to Fabric migrations.
Guidance for code conversion, storage path rewrites, Lakehouse integration, and orchestration replacement.
Cutover strategies, dual-writes validation, and end-to-end migration validation patterns.

Quick Start

Stand up a Fabric workspace and run a pilot migration to validate end-to-end parity.

Frequently Asked Questions about azure-hdinsight-migration-spark-to-fabric

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I migrate HDInsight Spark workloads to Microsoft Fabric?▼

Migrate HDInsight Spark workloads to Microsoft Fabric using an end-to-end playbook covering assessment, code rewrite, storage migration, metastore consolidation, and cutover validation. Requires access to the source HDInsight environment and a Fabric workspace with lakehouses.

What is the process for consolidating Hive Metastore during an HDInsight to Fabric migration?▼

Hive Metastore consolidation during HDInsight to Fabric migration involves unifying metastore schemas and tables into the Fabric Lakehouse. The migration playbook provides specific templates for mapping and transferring Hive metastore objects to OneLake.

Can I migrate HDInsight clusters from version 3.6 or 4.0 to Fabric Runtime 1.3?▼

Yes, the HDInsight to Fabric migration playbook supports clusters from the 3.6, 4.0, and 5.x eras. It validates compatibility and provides code conversion templates specifically targeting Fabric Runtime 1.3 or 2.0 environments.

What's the best way to validate Spark workload parity after migrating to a Fabric Lakehouse?▼

Validate Spark workload parity in a Fabric Lakehouse by implementing dual-writes validation and end-to-end cutover strategies. The migration playbook provides patterns to ensure data and processing consistency between the source and target environments.

Does the HDInsight to Fabric migration playbook include templates for replacing orchestration patterns?▼

Yes, the migration playbook includes specific templates for rewriting Spark, Hive, and orchestration patterns. It guides the replacement of existing HDInsight orchestration logic to execute natively within the Microsoft Fabric workspace environment.