skills.rest
patrikborosch
patrikboroschCOMMUNITY
Explore all skills by this author →
GitHub

spark-operations-cli

Diagnose failed Spark jobs and Livy sessions in Microsoft Fabric via read-only REST APIs.

14|3|Updated Mar 13, 2026
Search Tags:#performance tuning#microsoft fabric#livy sessions#spark diagnostics#job failure triage#pipeline run diagnosis
One-click install
npx skills add https://github.com/patrikborosch/AnalyticsPlatformAgents --skill spark-operations-cli-patrikborosch
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: spark-operations-cli
Source: https://github.com/patrikborosch/AnalyticsPlatformAgents/tree/main/creator/skills/spark-operations-cli
Command: npx skills add https://github.com/patrikborosch/AnalyticsPlatformAgents --skill spark-operations-cli-patrikborosch

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Manually debugging failed Spark jobs, stuck Livy sessions, and performance bottlenecks in Microsoft Fabric requires sifting through scattered logs, APIs, and UI tools, which is time-consuming and error-prone. This Skill automates the entire triage workflow to quickly identify root causes and actionable fixes.

Core Features & Use Cases

  • Read-Only Failure Triage: Diagnose failed Spark jobs, notebook runs, Lakehouse jobs, and pipeline Spark activities by retrieving logs, error details, and Spark Advisor findings without modifying cluster state.
  • Livy Session Health Checks: Detect stuck, idle, or zombie Livy sessions, assess resource utilization, and identify capacity pressure or configuration issues.
  • Performance Bottleneck Detection: Identify common performance anti-patterns including OOM errors, data skew, shuffle spill, GC pressure, and underutilized resources, with prioritized fix recommendations.
  • Use Case Example: If a production daily ETL notebook fails overnight, use this Skill to automatically resolve the workspace and notebook, pull the latest failed session's error details, and generate a report with the exact root cause (e.g., executor OOM from skewed data) and steps to fix it.

Quick Start

Use the spark-operations-cli skill to diagnose the root cause of your failed Fabric notebook 'ETL_Daily' and get a prioritized list of fixes to resolve the issue.

Frequently Asked Questions about spark-operations-cli

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose failed Spark jobs and notebook run failures in Microsoft Fabric?▼

Diagnose failed Spark jobs in Microsoft Fabric by using read-only REST API triage to retrieve logs, error details, and Spark Advisor findings without modifying cluster state. This automatically identifies root causes and actionable fixes for notebook run failures.

Why does my Fabric Livy session get stuck idle and how can I check its health?▼

Check stuck Livy sessions by assessing resource utilization and identifying capacity pressure or configuration issues. This detects zombie or idle sessions in Microsoft Fabric and highlights the underlying resource constraints causing the hang.

What is the best way to identify Spark performance bottlenecks like OOM errors and data skew?▼

Identify Spark performance bottlenecks like OOM errors, data skew, and shuffle spill by retrieving metrics from Fabric Spark Monitoring APIs. This approach pinpoints anti-patterns and GC pressure, returning prioritized recommendations to fix underutilized resources.

Can I troubleshoot pipeline Spark activity failures without modifying my Fabric cluster state?▼

Troubleshoot pipeline Spark activity failures safely using read-only Fabric control-plane APIs and Livy log endpoints. This retrieves diagnostic logs and root-cause recommendations without modifying cluster state or risking configuration drift.

Does this Spark triage process work for diagnosing Lakehouse job errors in Microsoft Fabric?▼

Yes, the Spark triage process works for diagnosing Lakehouse job errors in Microsoft Fabric. It retrieves specific error details and Spark Advisor findings for Lakehouse jobs to quickly resolve underlying execution failures.

How do I automatically resolve a failed Fabric notebook and get a prioritized list of fixes?▼

Automatically resolve a failed Fabric notebook by pulling the latest failed session's error details via REST APIs. This generates a report with the exact root cause, such as executor OOM from skewed data, and steps to fix it.

Related Skills

Explore More Data & Analytics Skills→
patrikboroschpatrikborosch
14

spark-consumption-cli

Run PySpark and Spark SQL analytics on Microsoft Fabric lakehouse data via Livy API sessions.

Community
Intermediate
rritecrritec
3

spark-operations-cli

Diagnoses failed Spark jobs and stuck Livy sessions using Fabric REST APIs and Spark Advisor data.

Community
Advanced
weiflycc-cmdweiflycc-cmd

spark-operations-cli

Diagnose failed Spark jobs and Livy sessions in Microsoft Fabric via read-only CLI.

Community
Advanced
microsoftmicrosoft
952

spark-operations-cli

Diagnose Spark job failures and Livy session health using read-only REST APIs.

Official
Advanced