skills.rest
rritec
rritecCOMMUNITY
Explore all skills by this author →
Website𝕏@rritecGitHub

hdinsight-migration

Migrate HDInsight Spark and Hive workloads to Microsoft Fabric.

3|6|Updated Jun 21, 2024
Search Tags:#delta lake#onelake#microsoft fabric#hdinsight#notebookutils#spark migration#oozie pipelines
One-click install
npx skills add https://github.com/rritec/Microsoft-Fabric --skill hdinsight-migration-rritec
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hdinsight-migration
Source: https://github.com/rritec/Microsoft-Fabric/tree/main/skills/plugins/fabric-skills/skills/hdinsight-migration
Command: npx skills add https://github.com/rritec/Microsoft-Fabric --skill hdinsight-migration-rritec

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you move legacy HDInsight Spark and Hive solutions into Microsoft Fabric by replacing outdated cluster patterns, storage paths, and orchestration models with Fabric-native equivalents.

Core Features & Use Cases

  • Converts legacy SparkContext, HiveContext, and SQLContext usage to the pre-instantiated SparkSession model in Fabric.
  • Rewrites WASB and ABFS storage paths into OneLake shortcuts and Fabric-friendly abfss or relative paths.
  • Translates Hive DDL, ORC tables, and Hive metastore patterns into Delta Lake tables and Lakehouse schemas.
  • Maps Oozie actions and coordinators to Fabric Pipelines, notebook activities, and schedule triggers.
  • Introduces notebookutils for file operations, secret retrieval, child notebook execution, and notebook orchestration during migration.

Quick Start

Ask me to migrate an HDInsight notebook, Hive table, or Oozie workflow to Microsoft Fabric, and I will turn it into Fabric-ready code and architecture guidance.

Frequently Asked Questions about hdinsight-migration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I migrate HDInsight Spark notebooks to Microsoft Fabric?▼

Migrating Oozie workflows to Fabric Pipelines involves mapping Oozie actions and coordinators to notebook activities and schedule triggers. This Skill refactors Oozie orchestration logic into Fabric-native Pipelines and utilizes notebookutils for child notebook execution and workflow scheduling.

What is the best way to convert Hive DDL to Delta Lake in Fabric?▼

No, HDInsight Spark workloads cannot run unmodified in Fabric because Fabric requires SparkSession-only execution and OneLake shortcuts. This Skill rewrites legacy cluster patterns, storage paths, and HiveContext usage into Fabric-native equivalents to ensure compatibility.

How do I convert WASB storage paths to OneLake shortcuts for Fabric?▼

Converting WASB storage paths to OneLake shortcuts requires rewriting them into Fabric-friendly abfss or relative paths. This Skill automates path conversion during migration to ensure your Spark workloads correctly reference OneLake storage in Microsoft Fabric.

Does HDInsight Spark work with Microsoft Fabric without code changes?▼

HDInsight Spark workloads require significant code changes for Fabric because Fabric mandates SparkSession-only execution and OneLake storage. This Skill automates the necessary refactoring of legacy SparkContext patterns and storage paths to produce Fabric-ready code.

Related Skills

Explore More Data & Analytics Skills→
weiflycc-cmdweiflycc-cmd

hdinsight-migration

Migrate HDInsight Spark, Hive, and Oozie workloads to Microsoft Fabric.

Community
Advanced
microsoftmicrosoft
952

hdinsight-migration

Migrate Azure HDInsight Spark and Hive workloads to Microsoft Fabric.

Official
Advanced
weiflycc-cmdweiflycc-cmd

spark-authoring-cli

Manage Microsoft Fabric Spark workloads and author PySpark, Scala, and SQL notebooks via REST API patterns.

Community
Advanced
bcastelinobcastelino
2

data-engineer

Design and implement scalable batch or streaming data pipelines.

Community
Advanced