skills.rest
rritec
rritecCOMMUNITY
Explore all skills by this author →
Website𝕏@rritecGitHub

e2e-medallion-architecture

Design and orchestrate end-to-end Medallion Architecture solutions in Microsoft Fabric.

3|6|Updated Jun 21, 2024
Search Tags:#pyspark#medallion architecture#pipeline orchestration#lakehouse#power bi#delta lake#microsoft fabric
One-click install
npx skills add https://github.com/rritec/Microsoft-Fabric --skill e2e-medallion-architecture-rritec
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: e2e-medallion-architecture
Source: https://github.com/rritec/Microsoft-Fabric/tree/main/skills/plugins/fabric-authoring/skills/e2e-medallion-architecture
Command: npx skills add https://github.com/rritec/Microsoft-Fabric --skill e2e-medallion-architecture-rritec

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It removes the complexity of designing and operating a complete Bronze/Silver/Gold lakehouse architecture in Microsoft Fabric, so data teams can move from raw ingestion to curated analytics with consistent governance.

Core Features & Use Cases

  • End-to-end medallion design: Plans separate Bronze, Silver, and Gold workspaces and lakehouses for strong isolation and ownership.
  • Notebook and pipeline orchestration: Guides notebook creation, lakehouse binding, sequential execution, and recurring pipeline automation across all layers.
  • Data quality and optimization: Enforces validation, deduplication, partition-aware writes, Delta Lake usage, and layer-specific Spark tuning.
  • Analytics consumption: Connects the Gold layer to Power BI through Direct Lake semantic models and reports.
  • Use Case: A data engineering team can use this Skill to turn a raw source feed into governed, validated dashboards without manually stitching together Fabric setup steps.

Quick Start

Ask the skill to design and execute a complete Microsoft Fabric medallion architecture for your project, including Bronze, Silver, and Gold workspaces, lakehouses, notebooks, validation, pipeline orchestration, and Power BI consumption.

Frequently Asked Questions about e2e-medallion-architecture

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a Bronze Silver Gold lakehouse architecture in Microsoft Fabric?▼

To build a Bronze Silver Gold lakehouse in Microsoft Fabric, create separate workspaces and lakehouses for each layer, use PySpark notebooks for sequential transformations, and enforce Delta Lake tables with validation checks before connecting Power BI.

What is medallion architecture and when should I use separate lakehouses in Fabric?▼

Medallion architecture organizes data into Bronze, Silver, and Gold layers for progressive refinement. Use separate Fabric lakehouses per layer to ensure strong isolation, clear ownership, and governed analytics across ingestion to curated consumption.

How do I orchestrate PySpark notebooks and pipelines across medallion architecture layers?▼

Orchestrate PySpark notebooks across medallion layers by binding each notebook to its target lakehouse, defining sequential execution for transformations, and scheduling recurring pipeline automation through Fabric for Bronze to Gold flows.

Do I need Delta Lake tables and workspace identifiers for Fabric medallion pipeline orchestration?▼

Yes, Delta Lake tables are required for storage and optimization across all medallion layers. Pipeline orchestration also relies on dynamically discovering workspace and item identifiers through the Fabric REST API to automate execution.

What's the best way to enforce data quality and Spark tuning in a Fabric medallion architecture?▼

The best way to enforce data quality in a Fabric medallion architecture is applying validation checks, deduplication, and partition-aware Delta Lake writes at each layer, combined with layer-specific Spark tuning for optimized Silver and Gold transformations.

Can I connect Power BI Direct Lake to a Gold lakehouse in Microsoft Fabric?▼

Yes, you can connect Fabric Gold lakehouse tables to Power BI through Direct Lake semantic models. This Skill guides the setup of Direct Lake reports to consume validated, curated Gold layer Delta Lake data for governed analytics.

Related Skills

Explore More Data & Analytics Skills→
weiflycc-cmdweiflycc-cmd

e2e-medallion-architecture

Implement end-to-end medallion architecture lakehouse patterns in Microsoft Fabric.

Community
Advanced
rbutinarrbutinar

powerbi-directlake-create

Create Power BI Semantic Models connected to Microsoft Fabric Lakehouse.

Community
Intermediate
weiflycc-cmdweiflycc-cmd

spark-authoring-cli

Manage Microsoft Fabric Spark workloads and author PySpark, Scala, and SQL notebooks via REST API patterns.

Community
Advanced
ThomazRossitoThomazRossito
31

Microsoft Fabric Lakehouse — Medallion Architecture

Design and implement a Medallion Lakehouse pattern in Microsoft Fabric.

Community
Advanced