synthetic-data-generation

Generate realistic synthetic data for Databricks using Python, Faker, and Spark.

Updated Mar 10, 2026

Search Tags:#data generation #synthetic data #faker #spark #parquet #databricks #unity catalog

Ready to run

npx skills add https://github.com/slysik/databricks-claude-coding --skill synthetic-data-generation-slysik

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: synthetic-data-generation
Source: https://github.com/slysik/databricks-claude-coding/tree/main/.agents/skills/synthetic-data-generation
Command: npx skills add https://github.com/slysik/databricks-claude-coding --skill synthetic-data-generation-slysik

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires faker, holidays, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the creation of realistic, story-driven synthetic data for Databricks, eliminating the need for manual data generation or the use of sensitive production data for testing and development.

Core Features & Use Cases

Realistic Data Generation: Creates data with non-linear distributions, integrity constraints, and time-based patterns using Faker and Spark.
Databricks Integration: Saves generated data as raw Parquet files to Unity Catalog Volumes, ready for downstream Spark Declarative Pipelines.
Use Case: Generate a synthetic customer, order, and ticket dataset for a Databricks demo environment, ensuring referential integrity and realistic attribute correlations.

Quick Start

Use the synthetic-data-generation skill to create 10,000 customer records and 50,000 order records in the 'my_catalog.my_schema' schema.

Frequently Asked Questions about synthetic-data-generation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I generate realistic synthetic data in Databricks with referential integrity?▼

You can generate synthetic data in Databricks that maintains referential integrity by using Python, Faker, and Spark to create datasets with non-linear distributions, integrity constraints, and row coherence, saving output as Parquet files to Unity Catalog Volumes.

Can I use Faker and Spark to create test data for Unity Catalog Volumes?▼

Yes, Faker and Spark can generate realistic test data and save it directly as raw Parquet files to Unity Catalog Volumes, providing believable distributions and time-based patterns ready for downstream Databricks pipelines.

What is the best way to create demo datasets for Databricks without using production data?▼

The best way to create Databricks demo datasets without production data is generating synthetic records with Python, Faker, and Spark, ensuring believable attribute correlations and referential integrity across tables.

Does synthetic data generation with Spark support non-linear distributions and time-based patterns?▼

Synthetic data generation with Spark supports non-linear distributions, time-based patterns, and integrity constraints, allowing you to generate story-driven datasets with realistic attribute correlations and row coherence for testing.

Do I need Python and Faker to generate synthetic Parquet files for Databricks?▼

Yes, this synthetic data generation approach uses Python and Faker alongside Spark to define realistic distributions and integrity constraints, outputting the results as raw Parquet files formatted for Unity Catalog Volumes.

Why use synthetic data generation for Spark Declarative Pipelines in Databricks?▼

Synthetic data generation provides realistic, story-driven Parquet datasets with referential integrity for Spark Declarative Pipelines, eliminating manual data creation and the need to expose sensitive production data during development.