skills.rest
9vantage
9vantageCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

hdinsight-migration

Migrates HDInsight Spark clusters, Hive workloads, and Oozie workflows to Microsoft Fabric.

Updated Jul 14, 2026
Search Tags:#migration#hive#spark#delta lake#microsoft fabric#hdinsight#oozie
One-click install
npx skills add https://github.com/9vantage/skills-for-fabric-clone --skill hdinsight-migration-9vantage
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hdinsight-migration
Source: https://github.com/9vantage/skills-for-fabric-clone/tree/main/plugins/fabric-skills/skills/hdinsight-migration
Command: npx skills add https://github.com/9vantage/skills-for-fabric-clone --skill hdinsight-migration-9vantage

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Retiring an Azure HDInsight cluster requires rewriting legacy Spark code, Hive DDL, storage paths, and Oozie orchestration for a new platform, which is error-prone and time-consuming when done manually. ## Core Features & Use Cases - Spark API Modernization: Replaces legacy HiveContext, SQLContext, and standalone SparkContext constructors with Fabric's pre-instantiated SparkSession. - Storage Path Conversion: Converts WASB and ABFS storage paths to OneLake abfss URLs via Shortcuts, eliminating storage account key configuration. - Hive to Delta Migration: Transforms Hive DDL (STORED AS ORC, external tables, partitioning) into Delta Lake schemas inside a Fabric Lakehouse. - Oozie Workflow Mapping: Maps Oozie actions (spark, hive, shell, sqoop, coordinator) to Fabric Data Pipeline activities and schedule triggers. - Use Case: When decommissioning an HDInsight cluster, use this Skill to convert all notebooks, Hive tables, and Oozie coordinators into Fabric notebooks, Lakehouse Delta tables, and scheduled Pipelines. ## Quick Start Migrate my HDInsight Spark notebooks and Hive tables to Microsoft Fabric, converting the WASB paths and Oozie workflows.

Frequently Asked Questions about hdinsight-migration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I migrate HDInsight Spark workloads to Microsoft Fabric?▼

Replace legacy HiveContext and SparkContext constructors with Fabric's pre-instantiated spark session, convert WASB paths to OneLake abfss URLs via Shortcuts, and rewrite Hive DDL as Delta Lake tables in a Lakehouse. Map Oozie workflows to Fabric Data Pipeline activities.

How to convert Hive DDL to Delta Lake tables in Fabric?▼

Replace STORED AS ORC or PARQUET clauses with USING DELTA, remove HDFS LOCATION paths for managed tables, and create a Lakehouse schema per Hive database using CREATE SCHEMA. Partitioned tables use Delta PARTITIONED BY syntax.

Does Microsoft Fabric support WASB storage paths?▼

No, wasb:// and wasbs:// paths are not supported in Fabric. Create a OneLake Shortcut to the existing Blob or ADLS Gen2 container, then reference data via OneLake abfss paths or relative Files/ paths in notebooks.

Can Oozie workflows run directly in Microsoft Fabric?▼

No, Oozie XML workflows must be mapped to Fabric Data Pipeline activities. Spark actions become Notebook or Spark Job Definition activities, sqoop becomes Copy Data, and coordinators become schedule or storage event triggers.

What replaces mssparkutils or dbutils when migrating from HDInsight?▼

Fabric provides notebookutils for file operations (notebookutils.fs.ls, cp), secret retrieval from Azure Key Vault (notebookutils.credentials.getSecret), and child notebook orchestration (notebookutils.notebook.run). HDInsight had no equivalent built-in utility framework.

Why does HiveContext fail in Microsoft Fabric notebooks?▼

HiveContext and SQLContext are legacy Spark 1.x/2.x APIs removed in Spark 3.x. Fabric notebooks pre-instantiate a SparkSession as spark, so calling these constructors raises errors; use spark.sql directly instead.

Related Skills

Explore More Data & Analytics Skills→
patrikboroschpatrikborosch
14

hdinsight-migration

Migrate HDInsight Spark, Hive, and Oozie workloads to Microsoft Fabric.

Community
Advanced
microsoftmicrosoft
952

hdinsight-migration

Migrate Azure HDInsight Spark and Hive workloads to Microsoft Fabric.

Official
Advanced
patrikboroschpatrikborosch
14

databricks-migration

Port Databricks notebooks, jobs, and Unity Catalog structures to Microsoft Fabric.

Community
Advanced
tomztomz

azure-hdinsight-migration-interactive-query-to-fabric-spark-or-dw

Migrate HDInsight Interactive Query workloads to Microsoft Fabric with schema and data compatibility.

Community
Advanced