Automate Hadoop cluster deployment and management on Azure HDInsight.
npx skills add https://github.com/tomz/agent-skills --skill azure-hdinsight-tomz
Please help me install this Agent Skill. Skill: azure-hdinsight Source: https://github.com/tomz/agent-skills/tree/main/azure-hdinsight Command: npx skills add https://github.com/tomz/agent-skills --skill azure-hdinsight-tomz
HDInsight clusters enable unified analytics on Azure, but provisioning, scaling, and securing them can be complex. This Skill consolidates guidance for deploying, configuring, and operating Hadoop ecosystem clusters (Spark, Hive LLAP, HBase, Kafka, Storm) and covers advanced topics like HDInsight on AKS, autoscale, and ESP migration.
Create a new Spark cluster on Azure HDInsight and submit a sample job to validate the setup.
High-intent search queries and answers about installing and using this skill.
Azure HDInsight cluster provisioning and autoscale are automated through CLI workflows for Spark, Hive LLAP, HBase, Kafka, and Storm. You can apply script actions to create, resize, and delete clusters while configuring autoscale settings for big data workloads.
HDInsight on AKS manages pool and cluster provisioning with multi-tenant deployment patterns and orchestration considerations. It extends standard Hadoop ecosystem operations by running clusters on Kubernetes infrastructure for unified analytics on Azure.
Enterprise security configurations and ESP migration for HDInsight clusters are supported through CLI-based provisioning steps. The process addresses ESP complexity directly to ensure production readiness and secure operational management.
The migration path from HDInsight to Fabric is included as part of operational guidance and gotchas. It provides best practices for transitioning existing Hadoop ecosystem clusters while maintaining production readiness.
Cluster management applies to Kafka, HBase, Hive LLAP, and Storm deployments on Azure. Common CLI workflows handle the creation, resizing, deletion, and lifecycle management for these Hadoop ecosystem components.
Known limitations include ESP complexity during enterprise security migration and specific production readiness requirements. Operational gotchas cover script action configurations, autoscale settings, and lifecycle management challenges across Spark and Kafka deployments.
BagelHoleConfigure Azure VNets, NSGs, and Private Endpoints via Azure CLI.
tomzMigrate HDInsight Spark workloads to Microsoft Fabric with an end-to-end playbook.
microsoftManage cryptographic keys in Azure using the Key Vault Keys SDK for JavaScript.
patrikboroschAutomate Azure Synapse Spark workload migration to Microsoft Fabric via REST APIs.
