e2e-medallion-architecture

Name: e2e-medallion-architecture
Availability: InStock
Author: patrikborosch

Automate Bronze/Silver/Gold medallion lakehouse pipelines in Microsoft Fabric.

14|3|Updated Mar 13, 2026

Search Tags:#data pipelines #pyspark #medallion architecture #lakehouse #power bi #delta lake #microsoft fabric

Ready to run

npx skills add https://github.com/patrikborosch/AnalyticsPlatformAgents --skill e2e-medallion-architecture-patrikborosch

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: e2e-medallion-architecture
Source: https://github.com/patrikborosch/AnalyticsPlatformAgents/tree/main/creator/skills/e2e-medallion-architecture
Command: npx skills add https://github.com/patrikborosch/AnalyticsPlatformAgents --skill e2e-medallion-architecture-patrikborosch

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Manually building and orchestrating multi-layer medallion (Bronze/Silver/Gold) lakehouse architectures in Microsoft Fabric is complex, error-prone, and often lacks proper data quality guardrails, layer isolation, and end-to-end validation from raw ingestion to analytics consumption.

Core Features & Use Cases

Full Medallion Lifecycle Automation: Automates setup of isolated Bronze, Silver, and Gold workspaces and lakehouses, ingestion, transformation, aggregation, and Power BI integration.
Built-in Data Quality & Optimization: Enforces deduplication, schema conformance, and layer-specific Spark optimizations (V-Order, ZORDER, partition-aware writes) for performance and reliability.
Use Case: For a retail analytics project, use this skill to automatically ingest daily sales CSVs into a Bronze layer, clean and conform data in Silver, build daily revenue aggregates in Gold, and connect a Power BI dashboard to the Gold layer for stakeholder reporting.

Quick Start

Use the e2e-medallion-architecture skill to set up a complete Bronze/Silver/Gold medallion lakehouse for sales analytics with automated pipelines and Power BI integration.

Frequently Asked Questions about e2e-medallion-architecture

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I build an end-to-end medallion lakehouse architecture in Microsoft Fabric?▼

To build a medallion lakehouse in Microsoft Fabric, you need to automate Bronze, Silver, and Gold layer isolation, enforce data quality, and orchestrate PySpark transformations ending in Power BI Direct Lake consumption. This handles multi-layer ingestion, sequential notebook execution, and semantic model creation.

What is the best way to orchestrate PySpark data pipelines across Delta Lake layers?▼

Orchestrating PySpark pipelines across Delta Lake layers requires sequential notebook execution and per-layer Spark optimizations like V-Order, ZORDER, and partition-aware writes. This approach ensures proper layer isolation and performance tuning from raw Bronze ingestion to Gold aggregation.

Can I use Microsoft Fabric REST API to deploy medallion architecture workspaces and lakehouses?▼

Yes, you can use the Fabric REST API to deploy isolated Bronze, Silver, and Gold workspaces and lakehouses automatically. This automates resource deployment alongside pipeline orchestration and semantic model creation for analytics reporting.

How does data quality enforcement work in a multi-layer Delta Lake medallion architecture?▼

Data quality enforcement in a Delta Lake medallion architecture works by applying deduplication and schema conformance during Silver layer transformations. This ensures reliable analytics before building daily aggregates in the Gold layer.

Does Power BI Direct Lake mode work with Gold layer Delta Lake tables in a Fabric lakehouse?▼

Yes, Power BI Direct Lake mode works with Gold layer Delta Lake tables by connecting dashboards directly to the Gold lakehouse for stakeholder reporting. This setup requires semantic model creation and proper layer isolation to function effectively.

Why should I separate Bronze, Silver, and Gold layers in a Fabric lakehouse?▼

Separating Bronze, Silver, and Gold layers in a Fabric lakehouse isolates raw data ingestion, transformation, and aggregation workloads. This separation enforces data quality guardrails and applies layer-specific Spark optimizations for reliable end-to-end analytics.