skills.rest
weiflycc-cmd
weiflycc-cmdCOMMUNITY
Explore all skills by this author →
GitHub

hdinsight-migration

Migrate HDInsight Spark, Hive, and Oozie workloads to Microsoft Fabric.

Updated Jun 1, 2026
Search Tags:#delta lake#onelake#microsoft fabric#hdinsight#spark migration#hive migration#oozie migration
One-click install
npx skills add https://github.com/weiflycc-cmd/skills-for-fabric --skill hdinsight-migration-weiflycc-cmd
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hdinsight-migration
Source: https://github.com/weiflycc-cmd/skills-for-fabric/tree/main/plugins/fabric-skills/skills/hdinsight-migration
Command: npx skills add https://github.com/weiflycc-cmd/skills-for-fabric --skill hdinsight-migration-weiflycc-cmd

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Migrating legacy Azure HDInsight Spark clusters, Hive data warehouses, and Oozie orchestration workflows to Microsoft Fabric requires extensive manual rewrites of deprecated Spark APIs, storage path formats, Hive DDL, and pipeline logic, creating significant overhead and risk of errors for data engineering teams.

Core Features & Use Cases

  • Legacy Spark API Modernization: Automatically replaces deprecated HDInsight Spark 1.x/2.x constructs including HiveContext, SQLContext, and standalone SparkContext() with Fabric's pre-instantiated SparkSession to eliminate runtime compatibility issues.
  • Storage and Schema Migration: Converts deprecated WASB/ABFS storage paths to OneLake abfss URLs or relative Lakehouse paths, and transforms Hive DDL (STORED AS ORC, external tables, partitioning) to Delta Lake schemas in Fabric Lakehouse.
  • Orchestration Workflow Mapping: Maps Oozie workflow actions (spark, hive, shell, sqoop, coordinator) to native Fabric Pipeline activities and schedule/storage event triggers, and introduces notebookutils for file system operations, secret retrieval, and child notebook orchestration previously handled via custom scripts or HDFS clients.
  • Use Case: Teams retiring HDInsight clusters can use this skill to accelerate migration of production Spark jobs, Hive data models, and scheduled workflows to Fabric without rewriting code from scratch.

Quick Start

Provide your HDInsight Spark notebook or Hive DDL script to the hdinsight-migration skill to automatically rewrite it for native compatibility with Microsoft Fabric's Spark runtime, OneLake storage, and Lakehouse schema structure.

Frequently Asked Questions about hdinsight-migration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I migrate Hive DDL to Delta Lake in Microsoft Fabric?▼

To migrate Hive DDL to Delta Lake, transform legacy Hive constructs like STORED AS ORC, external tables, and partitioning into native Delta Lake schemas within Fabric Lakehouse to ensure storage compatibility and query performance.

How do I convert WASB storage paths to OneLake for HDInsight migration?▼

Convert WASB storage paths to OneLake by replacing deprecated HDInsight storage references with native abfss URLs or relative Lakehouse paths to align HDInsight workloads with Microsoft Fabric's unified storage standard.

Can I map Oozie workflows to Fabric Pipelines?▼

Yes, map Oozie workflows to Fabric Pipelines by translating Oozie actions like spark, hive, shell, and sqoop into native Fabric Pipeline activities and schedule triggers to retain orchestration logic.

How do I update deprecated Spark APIs for Fabric compatibility?▼

Update deprecated Spark APIs by replacing legacy HDInsight Spark 1.x/2.x constructs like HiveContext and SQLContext with Fabric's pre-instantiated SparkSession to eliminate runtime compatibility issues during migration.

Do I need to rewrite HDFS file operations when moving to Microsoft Fabric?▼

No, you do not need to manually rewrite HDFS file operations because migrating to Fabric introduces notebookutils for file system operations, secret retrieval, and child notebook orchestration previously handled by custom scripts.

What is the best way to retire HDInsight clusters without rewriting Spark jobs?▼

The best way to retire HDInsight clusters without rewriting Spark jobs is applying automated migration logic that modernizes Spark APIs and converts storage paths to run production workloads natively on Fabric.

Related Skills

Explore More Data & Analytics Skills→
patrikboroschpatrikborosch
14

hdinsight-migration

Migrate HDInsight Spark, Hive, and Oozie workloads to Microsoft Fabric.

Community
Advanced
microsoftmicrosoft
952

hdinsight-migration

Migrate Azure HDInsight Spark and Hive workloads to Microsoft Fabric.

Official
Advanced
rritecrritec
3

hdinsight-migration

Migrate HDInsight Spark and Hive workloads to Microsoft Fabric.

Community
Advanced
patrikboroschpatrikborosch
14

databricks-migration

Port Databricks notebooks, jobs, and Unity Catalog structures to Microsoft Fabric.

Community
Advanced