skills.rest
PatrickGallucci
PatrickGallucciCOMMUNITY
Explore all skills by this author →
GitHub

fabric-delta-spark-perf

Diagnose Delta Lake and Spark performance bottlenecks in Fabric Lakehouses.

16|1|Updated Feb 10, 2026
Search Tags:#optimization#diagnostics#spark#fabric#delta-lake#vacuum#vorder
One-click install
npx skills add https://github.com/PatrickGallucci/fabric-skills --skill fabric-delta-spark-perf
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: fabric-delta-spark-perf
Source: https://github.com/PatrickGallucci/fabric-skills/tree/main/skills/fabric-delta-spark-perf
Command: npx skills add https://github.com/PatrickGallucci/fabric-skills --skill fabric-delta-spark-perf

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyspark, delta-spark, and includes scripts (resource) and references (resource) components.

What problem does it solve?

Delta Lake and Spark performance can degrade over time in Fabric Lakehouses due to small files, data skew, and suboptimal configurations. This Skill provides a structured diagnostic workflow and actionable remediation to restore throughput and reduce latency.

Core Features & Use Cases

  • End-to-end diagnostics for Delta Lake on Fabric: file health checks, partition assessment, data skew analysis, and Spark config audit.
  • Recommended optimizations: OPTIMIZE, VACUUM, V-Order / Z-Order, Optimized Write, and AQE tuning, plus resource-profile guidance.
  • Real-world scenarios: slow Spark notebooks, heavy shuffle workloads, and Direct Lake-backed tables.

Quick Start

Activate the skill and run the diagnostic against your Delta Lake table or schema to identify issues, then apply recommended optimizations such as OPTIMIZE, VACUUM, and AQE tuning.

Frequently Asked Questions about fabric-delta-spark-perf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose slow Spark jobs in a Fabric Lakehouse?▼

Use the diagnostic workflow to audit file health, assess partitions, analyze data skew, and review Spark configs to identify bottlenecks causing slow Delta Lake and Spark jobs in your Fabric Lakehouse.

What is the best way to fix small file proliferation in Delta Lake?▼

Fix small file proliferation in Delta Lake by applying the OPTIMIZE workflow to compact small files, followed by VACUUM to remove unreferenced data files and restore table read throughput.

When should I use V-Order or Z-Order optimization on Delta tables?▼

Apply V-Order optimization to improve Direct Lake read performance, and use Z-Order optimization to accelerate filtered queries on high-cardinality columns within your Delta Lake tables.

Does this Delta Lake optimization approach support Adaptive Query Execution tuning?▼

Yes, the diagnostic workflow supports Adaptive Query Execution (AQE) tuning, providing configuration guidance to reduce excessive shuffles and dynamically manage heavy workloads in Spark.

How do I remediate data skew and excessive shuffles in Spark?▼

Remediate data skew and excessive shuffles by auditing Spark resource profiles, applying Adaptive Query Execution tuning, and using Z-Order optimization to balance data distribution across partitions.

Can I use pyspark and delta-spark for Fabric Lakehouse maintenance?▼

Yes, you can use pyspark and delta-spark dependencies to execute script-based maintenance utilities that run OPTIMIZE and VACUUM workflows for Fabric Lakehouse Delta tables.

Related Skills

Explore More Data & Analytics Skills→
microsoftmicrosoft
952

spark-consumption-cli

Run interactive PySpark and Spark SQL queries on Fabric Lakehouse Livy sessions.

Official
Advanced
WbunkerWbunker

delta-lake-expert

Implement Delta Lake tables and tune performance with liquid clustering and Z-ordering.

Community
Advanced
PatrickGallucciPatrickGallucci
16

fabric-notebook-perf-remediate

Diagnose and remediate Apache Spark performance bottlenecks in Microsoft Fabric notebooks.

Community
Advanced
PatrickGallucciPatrickGallucci
16

fabric-spark-perf-remediate

Diagnose and remediate Fabric Spark performance problems across workloads.

Community
Advanced