skills.rest
patrikborosch
patrikboroschCOMMUNITY
Explore all skills by this author →
GitHub

spark-authoring-cli

Automate Microsoft Fabric Spark artifact provisioning and notebook code authoring via REST API.

14|3|Updated Mar 13, 2026
Search Tags:#data engineering#spark#pyspark#delta lake#microsoft fabric#notebook authoring#materialized lake view
One-click install
npx skills add https://github.com/patrikborosch/AnalyticsPlatformAgents --skill spark-authoring-cli-patrikborosch
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: spark-authoring-cli
Source: https://github.com/patrikborosch/AnalyticsPlatformAgents/tree/main/creator/skills/spark-authoring-cli
Command: npx skills add https://github.com/patrikborosch/AnalyticsPlatformAgents --skill spark-authoring-cli-patrikborosch

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill eliminates the tedious manual work of managing Microsoft Fabric Spark artifacts (workspaces, lakehouses, notebooks, jobs) and writing inconsistent, unoptimized Spark notebook code, reducing development time and ensuring adherence to Fabric and Delta Lake best practices for data engineering workloads.

Core Features & Use Cases

  • Fabric Artifact Management: Automate provisioning and configuration of workspaces, lakehouses, notebooks, and Spark jobs via REST API, with built-in validation for capacity assignment, idempotency, and duplicate job prevention.
  • Notebook Code Authoring: Generate production-ready PySpark, Scala, SparkR, and SQL notebook code for Fabric, with correct lakehouse bindings, Spark session configuration, and medallion architecture patterns.
  • Materialized Lake View (MLV) Support: Author, review, and optimize MLVs for incremental refresh, including data quality constraints, layering patterns, and refresh-readiness validation.
  • Use Case Example: A data engineer building a new analytics pipeline on Fabric can use this Skill to set up a dev workspace and lakehouse, generate a parameterized bronze ingestion notebook with explicit schemas and error handling, and create a silver MLV for cleaned customer data optimized for incremental refresh.

Quick Start

Use the spark-authoring-cli skill to provision a new Fabric dev workspace and lakehouse, then generate a PySpark notebook for ingesting public CSV data into a Delta table with explicit schema and data quality validation.

Frequently Asked Questions about spark-authoring-cli

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate Microsoft Fabric workspace and lakehouse provisioning?▼

To automate Fabric workspace and lakehouse provisioning, apply REST API calls for idempotent infrastructure operations. This validates capacity assignment and prevents duplicate artifact creation, replacing manual configuration with reliable, automated provisioning.

What is the best way to generate production-ready PySpark notebooks for Fabric?▼

Generating production-ready PySpark notebooks for Fabric is best achieved through automated authoring that enforces Delta Lake best practices. This guarantees correct lakehouse bindings, explicit schema definitions, and proper Spark session configuration for medallion architecture patterns.

How do I set up Materialized Lake Views for incremental refresh in Fabric?▼

Setting up Materialized Lake Views for incremental refresh in Fabric requires authoring with data quality constraints and layering patterns. You must then validate refresh-readiness to ensure correct Delta Lake optimization and successful scheduled updates.

Can I use Scala and SQL alongside PySpark for Fabric notebook development?▼

Yes, you can use Scala, SQL, and SparkR alongside PySpark for Fabric notebook development. Automated authoring generates production-ready code across these languages while maintaining correct lakehouse bindings and consistent Spark session configuration.

How do I orchestrate Spark jobs using the Microsoft Fabric REST API?▼

Orchestrating Spark jobs via the Microsoft Fabric REST API requires correct payload formatting for control-plane and data-plane interactions. Proper Fabric API authentication is necessary to ensure idempotent operations and prevent duplicate job execution.

Related Skills

Explore More Data & Analytics Skills→
patrikboroschpatrikborosch
14

eventstream-consumption-cli

List, inspect, and monitor Microsoft Fabric Eventstream pipelines via the REST API.

Community
Intermediate
wardawgmalviciouswardawgmalvicious
2

fabric-mlv

Automates Materialized Lake View creation, management and refresh in Microsoft Fabric using Spark/PySpark.

Community
Advanced
erichowenserichowens
181

data-pipeline-engineer

Automate ETL/ELT pipeline design, validation, and operation across medallion architectures.

Community
Advanced
rritecrritec
3

spark-authoring-cli

Manage Microsoft Fabric Spark authoring tasks and generate notebook-ready code.

Community
Advanced