data-engineering-patterns-fabric-databricks

Provide data engineering patterns for Azure Databricks, Microsoft Fabric, and PySpark.

5|1|Updated May 16, 2026

Search Tags:#best practices #patterns #data engineering #pipeline design #PySpark #Microsoft Fabric #Azure Databricks

Ready to run

npx skills add https://github.com/Aradotso/data-skills --skill data-engineering-patterns-fabric-databricks

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: data-engineering-patterns-fabric-databricks
Source: https://github.com/Aradotso/data-skills/tree/main/skills/data-engineering-patterns-fabric-databricks
Command: npx skills add https://github.com/Aradotso/data-skills --skill data-engineering-patterns-fabric-databricks

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires git, python, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides data engineers with a comprehensive library of patterns for designing and implementing scalable, reliable, and cost-effective data pipelines, architectures, and systems.

Core Features & Use Cases

Patterns for Microsoft Fabric: 250 patterns cover pipeline design, lakehouse architecture, warehouse and SQL, Power BI, and more.
Patterns for Azure Databricks: 350 patterns include cluster management, Delta Lake, workflows, and cost optimization.
Patterns for PySpark: 88 concepts for production Spark across both platforms.
Use Case: Utilize this Skill to optimize your Azure Databricks and Microsoft Fabric clusters, implement best practices for Delta Lake, and design robust PySpark transformations.

Quick Start

Clone the repository to access the pattern PDFs: git clone https://github.com/ssanjaychandra123/data-engineering-patterns.git and navigate to data-engineering-patterns/.

Frequently Asked Questions about data-engineering-patterns-fabric-databricks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

What are the best practices for PySpark data engineering in production?▼

PySpark data engineering best practices include 88 production concepts covering scalable pipeline design and robust transformations across Azure Databricks and Microsoft Fabric platforms.

How do I optimize cluster management and costs in Azure Databricks?▼

Azure Databricks cost optimization involves applying 350 specific patterns for cluster management, Delta Lake implementation, and workflow orchestration to ensure scalable and cost-effective systems.

Do I need Git and Python to use these data engineering patterns?▼

Yes, Git and Python are required to clone the pattern repository and apply the data engineering concepts to your Azure Databricks and Microsoft Fabric environments.

What patterns are available for Microsoft Fabric lakehouse architecture?▼

Microsoft Fabric lakehouse architecture patterns include 250 concepts covering pipeline design, warehouse and SQL optimization, and Power BI integration for reliable data systems.

How do I implement Delta Lake best practices for scalable data pipelines?▼

Delta Lake best practices are implemented using Azure Databricks patterns that address scalable data pipeline design, lakehouse architecture, and cluster cost optimization.

Can I use these patterns for both Microsoft Fabric and Azure Databricks?▼

Yes, the patterns cover 600+ concepts for Microsoft Fabric, Azure Databricks, and PySpark, supporting pipeline design, warehouse management, and data transformations across both platforms.