skills.rest
rritec
rritecCOMMUNITY
Explore all skills by this author →
Website𝕏@rritecGitHub

spark-consumption-cli

Analyze Microsoft Fabric lakehouse data with interactive PySpark sessions and Spark SQL.

3|6|Updated Jun 21, 2024
Search Tags:#pyspark#dataframe#fabric#lakehouse#spark-sql#livy#delta-time-travel
One-click install
npx skills add https://github.com/rritec/Microsoft-Fabric --skill spark-consumption-cli-rritec
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: spark-consumption-cli
Source: https://github.com/rritec/Microsoft-Fabric/tree/main/skills/plugins/fabric-skills/skills/spark-consumption-cli
Command: npx skills add https://github.com/rritec/Microsoft-Fabric --skill spark-consumption-cli-rritec

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill removes the friction of interactive Spark analysis in Microsoft Fabric by guiding users through Lakehouse Livy sessions, workspace discovery, and PySpark-based exploration.

Core Features & Use Cases

  • Interactive Spark Sessions: Create, reuse, and clean up Lakehouse Livy sessions for hands-on analysis without unnecessary resource waste.
  • Advanced Lakehouse Analytics: Work with Spark DataFrames, Spark SQL, cross-lakehouse joins, Delta time travel, schema inspection, and JSON or unstructured data.
  • Fabric-Specific Operations: Use the correct Fabric REST and OneLake access patterns, token audiences, pagination, and long-running operation handling for reliable execution.

Quick Start

Ask the assistant to analyze your Fabric lakehouse data with PySpark and Spark SQL, and to create or reuse an interactive Livy session for the target workspace and lakehouse.

Frequently Asked Questions about spark-consumption-cli

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I analyze Fabric lakehouse data with PySpark and Spark SQL?▼

You can analyze Fabric lakehouse data by creating interactive Livy sessions to run PySpark and Spark SQL. This approach handles workspace discovery, cross-lakehouse joins, JSON exploration, data quality checks, and Delta time-travel analysis.

How does Livy session lifecycle handling work for Fabric lakehouse analytics?▼

Livy session lifecycle handling creates, reuses, and cleans up interactive Spark sessions. This prevents unnecessary resource waste during PySpark analysis while ensuring reliable execution for long-running operations and pagination.

Can I perform Delta time-travel analysis on Microsoft Fabric lakehouses?▼

Yes, Delta time-travel analysis is supported. You can query previous versions of your lakehouse data using PySpark and Spark SQL, alongside schema inspection and exploration of JSON or unstructured data.

What's the best way to explore JSON and unstructured data in a Fabric lakehouse?▼

The best way to explore JSON and unstructured data is through interactive PySpark sessions using Livy. This method applies Fabric-specific REST access patterns, pagination, and long-running operation handling for reliable data exploration.

Related Skills

Explore More Data & Analytics Skills→
patrikboroschpatrikborosch
14

spark-consumption-cli

Run PySpark and Spark SQL analytics on Microsoft Fabric lakehouse data via Livy API sessions.

Community
Intermediate
microsoftmicrosoft
952

spark-consumption-cli

Run interactive PySpark and Spark SQL queries on Fabric Lakehouse Livy sessions.

Official
Advanced
weiflycc-cmdweiflycc-cmd

spark-consumption-cli

Run interactive PySpark and Spark SQL analytics on Microsoft Fabric lakehouse data via the Lakehouse Livy API.

Community
Intermediate
PatrickGallucciPatrickGallucci
16

fabric-spark-compute-perf-remediate

Diagnose and remediate performance issues across Fabric Spark compute and Delta Lake tables.

Community
Advanced