skills.rest
Aradotso
AradotsoOFFICIAL
Explore all skills by this author →
Website𝕏@aradotsoGitHub

data-engineering-patterns-fabric-databricks

Provide data engineering patterns for Azure Databricks, Microsoft Fabric, and PySpark.

5|1|Updated May 16, 2026
Search Tags:#best practices#patterns#data engineering#pipeline design#PySpark#Microsoft Fabric#Azure Databricks
One-click install
npx skills add https://github.com/Aradotso/data-skills --skill data-engineering-patterns-fabric-databricks
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-engineering-patterns-fabric-databricks
Source: https://github.com/Aradotso/data-skills/tree/main/skills/data-engineering-patterns-fabric-databricks
Command: npx skills add https://github.com/Aradotso/data-skills --skill data-engineering-patterns-fabric-databricks

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires git, python, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides data engineers with a comprehensive library of patterns for designing and implementing scalable, reliable, and cost-effective data pipelines, architectures, and systems.

Core Features & Use Cases

  • Patterns for Microsoft Fabric: 250 patterns cover pipeline design, lakehouse architecture, warehouse and SQL, Power BI, and more.
  • Patterns for Azure Databricks: 350 patterns include cluster management, Delta Lake, workflows, and cost optimization.
  • Patterns for PySpark: 88 concepts for production Spark across both platforms.
  • Use Case: Utilize this Skill to optimize your Azure Databricks and Microsoft Fabric clusters, implement best practices for Delta Lake, and design robust PySpark transformations.

Quick Start

Clone the repository to access the pattern PDFs: git clone https://github.com/ssanjaychandra123/data-engineering-patterns.git and navigate to data-engineering-patterns/.

Frequently Asked Questions about data-engineering-patterns-fabric-databricks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What are the best practices for PySpark data engineering in production?▼

PySpark data engineering best practices include 88 production concepts covering scalable pipeline design and robust transformations across Azure Databricks and Microsoft Fabric platforms.

How do I optimize cluster management and costs in Azure Databricks?▼

Azure Databricks cost optimization involves applying 350 specific patterns for cluster management, Delta Lake implementation, and workflow orchestration to ensure scalable and cost-effective systems.

Do I need Git and Python to use these data engineering patterns?▼

Yes, Git and Python are required to clone the pattern repository and apply the data engineering concepts to your Azure Databricks and Microsoft Fabric environments.

What patterns are available for Microsoft Fabric lakehouse architecture?▼

Microsoft Fabric lakehouse architecture patterns include 250 concepts covering pipeline design, warehouse and SQL optimization, and Power BI integration for reliable data systems.

How do I implement Delta Lake best practices for scalable data pipelines?▼

Delta Lake best practices are implemented using Azure Databricks patterns that address scalable data pipeline design, lakehouse architecture, and cluster cost optimization.

Can I use these patterns for both Microsoft Fabric and Azure Databricks?▼

Yes, the patterns cover 600+ concepts for Microsoft Fabric, Azure Databricks, and PySpark, supporting pipeline design, warehouse management, and data transformations across both platforms.

Related Skills

Explore More Data & Analytics Skills→
vaughnangoyvaughnangoy

clone-repo

Clone GitHub and other git repositories into a structured local directory layout.

Community
Advanced
abdullah1854abdullah1854
1

fabric-helper

Generate Delta Lake and Spark SQL queries for Fabric Lakehouse and Warehouse.

Community
Intermediate
patrikboroschpatrikborosch
14

sqldw-operations-cli

Diagnose Microsoft Fabric Data Warehouse performance bottlenecks with read-only T-SQL queries.

Community
Intermediate
bcastelinobcastelino
2

data-engineering-data-pipeline

Design and implement batch and streaming data pipelines with Airflow, dbt, and Delta Lake.

Community
Advanced