skills.rest
ice-droid-99
ice-droid-99COMMUNITY
Explore all skills by this author →
GitHub

databricks-synthetic-data-gen

Generate scalable synthetic datasets in Databricks using Spark and Faker.

Updated May 20, 2026
Search Tags:#synthetic data#faker#spark#databricks#unity catalog#parquet generation#ml-ready datasets
One-click install
npx skills add https://github.com/ice-droid-99/BI-Dashboard-Generation-Agent-in-Databricks- --skill databricks-synthetic-data-gen-ice-droid-99
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-synthetic-data-gen
Source: https://github.com/ice-droid-99/BI-Dashboard-Generation-Agent-in-Databricks-/tree/main/.gemini/skills/databricks-synthetic-data-gen
Command: npx skills add https://github.com/ice-droid-99/BI-Dashboard-Generation-Agent-in-Databricks- --skill databricks-synthetic-data-gen-ice-droid-99

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill removes the effort of creating believable datasets by generating scalable, story-driven synthetic data that works for Databricks dashboards and ML experiments.

Core Features & Use Cases

  • Business-story synthetic tables: Produces coherent, linked master/child datasets (e.g., incidents → ticket spikes → churn and revenue impact) so the data has something meaningful to analyze.
  • Scalable Databricks generation: Uses Spark + Faker with pandas UDFs for parallel generation from thousands to millions of rows.
  • Flexible outputs & integrity: Writes Parquet (default) to Unity Catalog volumes or creates Delta tables when requested, while encouraging referential-integrity-friendly generation patterns.
  • Safety guardrails: Enforces user-supplied catalog/schema, requires plan approval before code generation, and avoids serverless-incompatible patterns like cache/persist.

Quick Start

Ask it to generate a synthetic support incident dataset for your demo and tell it the Unity Catalog and schema you want to use.

Frequently Asked Questions about databricks-synthetic-data-gen

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate synthetic data in Databricks for dashboarding and ML analysis?▼

This Skill creates business-story synthetic datasets by generating linked master and child tables with coherent relationships. It produces data with non-uniform distributions over time, enabling meaningful anomaly and business impact analysis.

Can I write synthetic data directly to Delta tables in Unity Catalog?▼

Yes, you can write synthetic data to Delta tables in Unity Catalog. The default output is Parquet on Unity Catalog volumes, but it creates Delta tables when specifically requested by supplying your target catalog and schema.

Does synthetic data generation with Faker work on Databricks serverless compute?▼

Yes, synthetic data generation with Faker works on Databricks serverless compute. The execution avoids serverless-incompatible patterns like cache and persist, ensuring scalable partitioning through pandas UDFs.

How do I scale Faker test data generation to millions of rows in Spark?▼

You scale Faker data generation by distributing the workload across Spark partitions using pandas UDFs. This parallel execution approach supports scalable generation from thousands to millions of rows.

What are the limitations of generating synthetic datasets in Databricks?▼

Limitations include the requirement to supply a specific Unity Catalog and schema before generation, and the necessity of plan approval before any write code is produced. It also encourages but does not strictly enforce referential integrity patterns.

Related Skills

Explore More Data & Analytics Skills→
PaldomPaldom
11

databricks-synthetic-data-gen

Generate synthetic data with Spark and Faker into Unity Catalog volumes.

Community
Advanced
aaronachermannaaronachermann

databricks-synthetic-data-gen

Generate story-driven synthetic datasets for Databricks with Spark and Faker UDFs.

Community
Advanced
tomztomz

azure-databricks

Provision Azure Databricks workspaces with Unity Catalog governance via az CLI.

Community
Advanced
thbehthbeh

databricks-synthetic-data-generation

Generate synthetic datasets with Faker and store them in Databricks volumes.

Community
Intermediate