skills.rest
aaronachermann
aaronachermannCOMMUNITY
Explore all skills by this author →
GitHub

databricks-synthetic-data-gen

Generate story-driven synthetic datasets for Databricks with Spark and Faker UDFs.

Updated Apr 18, 2026
Search Tags:#testing#faker#spark#data-generation#databricks#unity-catalog#synthetic-data
One-click install
npx skills add https://github.com/aaronachermann/PolentaEncoders --skill databricks-synthetic-data-gen-aaronachermann
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-synthetic-data-gen
Source: https://github.com/aaronachermann/PolentaEncoders/tree/main/.github/skills/databricks-synthetic-data-gen
Command: npx skills add https://github.com/aaronachermann/PolentaEncoders --skill databricks-synthetic-data-gen-aaronachermann

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires faker, pandas, numpy, databricks-connect, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps teams quickly create realistic synthetic datasets for Databricks to test pipelines, run demos, and validate analytics without exposing real customer data.

Core Features & Use Cases

  • Spark + Faker data generation for realistic, story-driven datasets
  • Serverless-ready and scalable to millions of rows
  • Multiple output formats (Parquet/Delta/JSON/CSV) and Unity Catalog friendly schemas
  • Data storytelling patterns with referential integrity and measurable business impact

Quick Start

Configure the data story you want (for example a 100000-row customers and orders dataset with skewed tiers and a simulated outage) and run the generator to produce raw data in your catalog volume.

Frequently Asked Questions about databricks-synthetic-data-gen

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate realistic synthetic data in Databricks for testing pipelines?▼

Generate realistic synthetic data in Databricks using Spark and Faker UDFs to create story-driven datasets with non-uniform distributions and referential integrity. This allows you to test pipelines and run demos without exposing real customer data.

Can I create synthetic datasets with referential integrity for Databricks dashboards?▼

Yes, you can create synthetic datasets with referential integrity across customers, orders, and incidents. The generator enforces relationships and measurable business impacts, making the output suitable for dashboards and ML models.

Does synthetic data generation for Databricks support Unity Catalog integration?▼

Yes, synthetic data generation for Databricks supports Unity Catalog integration with planning prompts and friendly schemas. It outputs raw data directly to your catalog volume in Delta, Parquet, JSON, or CSV formats.

What is the best way to scale synthetic data generation to millions of rows on Databricks?▼

The best way to scale synthetic data generation to millions of rows on Databricks is using serverless-ready Spark and Faker UDFs. This approach efficiently produces large datasets while maintaining skewed distributions and story-driven patterns.

Do I need Databricks-connect to run the synthetic data generator?▼

Yes, you need databricks-connect along with faker, pandas, and numpy dependencies to run the synthetic data generator. These libraries enable serverless execution and the creation of story-driven datasets for root-cause analysis.

Related Skills

Explore More Data & Analytics Skills→
leary-pokenleary-poken
1

databricks-synthetic-data-gen

Generate scalable synthetic data for Databricks workloads using Spark and Faker.

Community
Advanced
ThomazRossitoThomazRossito
31

databricks-synthetic-data-gen

Generate story-driven synthetic datasets in Databricks with Spark and Faker.

Community
Advanced
PaldomPaldom
11

databricks-synthetic-data-gen

Generate synthetic data with Spark and Faker into Unity Catalog volumes.

Community
Advanced
alessandro9110alessandro9110
4

databricks-synthetic-data-generation

Generate synthetic transactional data with Faker and Spark for Databricks.

Community
Advanced