skills.rest
rritec
rritecCOMMUNITY
Explore all skills by this author →
Website𝕏@rritecGitHub

spark-operations-cli

Diagnoses failed Spark jobs and stuck Livy sessions using Fabric REST APIs and Spark Advisor data.

3|6|Updated Jun 21, 2024
Search Tags:#performance analysis#spark#microsoft fabric#pipeline diagnosis#livy#spark advisor
One-click install
npx skills add https://github.com/rritec/Microsoft-Fabric --skill spark-operations-cli-rritec
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: spark-operations-cli
Source: https://github.com/rritec/Microsoft-Fabric/tree/main/skills/plugins/fabric-operations/skills/spark-operations-cli
Command: npx skills add https://github.com/rritec/Microsoft-Fabric --skill spark-operations-cli-rritec

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps you quickly diagnose failed Microsoft Fabric Spark jobs, stuck Livy sessions, and slow workloads without changing anything in the environment.

Core Features & Use Cases

  • Failure Triage: Identify root causes for notebook runs, Spark Job Definitions, and Lakehouse jobs using session state, Spark Advisor findings, logs, and job history.
  • Performance Analysis: Detect data skew, shuffle spill, GC pressure, underutilized capacity, and other bottlenecks using monitoring APIs and resource metrics.
  • Pipeline Diagnosis: Trace Spark-related failures inside pipeline runs, extract session details, and report the exact failed activity and traceback when available.
  • Offline Escalation: Copy event logs and open a local Spark History Server when online monitoring data is incomplete or expired.

Quick Start

Ask: Diagnose my failed Fabric notebook in workspace Production and explain the root cause with the most important fix.

Frequently Asked Questions about spark-operations-cli

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose failed Microsoft Fabric Spark jobs and stuck Livy sessions?▼

Triage failed Fabric Spark jobs by querying Fabric REST APIs and az rest to extract session state, Spark Advisor findings, and job history for notebook runs, Spark Job Definitions, and Lakehouse table loads.

What causes slow Microsoft Fabric Spark workloads and how can I detect performance bottlenecks?▼

Detect Spark performance bottlenecks such as data skew, shuffle spill, and GC pressure by analyzing monitoring APIs and resource usage metrics to identify underutilized capacity and slow workloads.

How do I trace Spark-related failures inside Fabric pipeline activity runs?▼

Trace Spark failures inside Fabric pipeline runs by extracting session details and reporting the exact failed activity and traceback from read-only monitoring data and pipeline activity run history.

Can I diagnose Fabric Spark notebook failures if monitoring data is incomplete or expired?▼

Yes, when online monitoring data is incomplete or expired, you can copy event logs and open a local Spark History Server as a fallback to continue diagnosing the Fabric Spark failure offline.

Do I need write access to the environment to triage Fabric Spark job failures?▼

No, diagnosing Fabric Spark failures requires only read-only monitoring data from Fabric REST APIs, Spark Advisor, and log retrieval, so no environment changes or write access are required.

Related Skills

Explore More Data & Analytics Skills→
weiflycc-cmdweiflycc-cmd

spark-operations-cli

Diagnose failed Spark jobs and Livy sessions in Microsoft Fabric via read-only CLI.

Community
Advanced
microsoftmicrosoft
952

spark-operations-cli

Diagnose Spark job failures and Livy session health using read-only REST APIs.

Official
Advanced
patrikboroschpatrikborosch
14

spark-operations-cli

Diagnose failed Spark jobs and Livy sessions in Microsoft Fabric via read-only REST APIs.

Community
Advanced
yangyongyongyongyangyongyongyong
1

spark-k8s

Diagnose Spark jobs and tasks on Kubernetes via REST APIs.

Community
Advanced