skills.rest
teegin-g
teegin-gCOMMUNITY
Explore all skills by this author →
GitHub

databricks-synthetic-data-gen

Generate scalable synthetic data for Databricks using Spark and Faker.

Updated Jan 30, 2026
Search Tags:#generate#faker#spark#data-generation#databricks#synthetic-data
One-click install
npx skills add https://github.com/teegin-g/Slopcast --skill databricks-synthetic-data-gen-teegin-g
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-synthetic-data-gen
Source: https://github.com/teegin-g/Slopcast/tree/main/.agents/skills/databricks-synthetic-data-gen
Command: npx skills add https://github.com/teegin-g/Slopcast --skill databricks-synthetic-data-gen-teegin-g

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires faker, pandas, numpy, and includes scripts (resource) and references (resource) components.

What problem does it solve?

Generate realistic, scalable synthetic datasets for Databricks workloads using Spark + Faker, enabling fast testing, demos, and data science experiments without exposing real data.

Core Features & Use Cases

  • Spark + Faker UDFs for distributed data generation
  • Serverless and classic compute compatibility with multiple output formats (Parquet/JSON/CSV/Delta)
  • Master-detail data generation with referential integrity and volume-ready pipelines
  • Ideal for testing, sandbox analytics, and demonstration datasets in Unity Catalog volumes

Quick Start

Run the script to generate synthetic data in a Unity Catalog volume using Spark + Faker.

Frequently Asked Questions about databricks-synthetic-data-gen

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate synthetic data in Databricks using Spark?▼

Spark Faker UDFs generate scalable synthetic data for Databricks by distributing row creation across worker nodes. This handles thousands to millions of rows and writes directly to Unity Catalog volumes in Parquet, JSON, CSV, or Delta formats.

Can I generate master-detail datasets with referential integrity in Databricks?▼

Yes, this master-detail data generation supports referential integrity across parent-child records. It applies non-linear distributions and time-based patterns to produce realistic relational structures for testing and demonstrations within Unity Catalog volumes.

Does synthetic data generation work with Databricks serverless compute?▼

Yes, this synthetic data generation supports both serverless and classic Databricks compute. It leverages Spark and Faker UDFs to execute distributed generation workflows and outputs results directly to Unity Catalog volumes.

What is the best way to create large test datasets without exposing real data?▼

Using Spark with Faker UDFs provides the best way to create large synthetic datasets. It generates realistic rows with non-linear distributions, protecting real data while accelerating testing and data science experiments at scale.

Do I need Unity Catalog volumes to store generated synthetic data?▼

Yes, the generated synthetic data writes directly to Unity Catalog volumes. Storing data in these volumes allows you to securely manage sandbox analytics and demonstration datasets within your Databricks environment.

Related Skills

Explore More Data & Analytics Skills→
PaldomPaldom
11

databricks-synthetic-data-gen

Generate synthetic data with Spark and Faker into Unity Catalog volumes.

Community
Advanced
aaronachermannaaronachermann

databricks-synthetic-data-gen

Generate story-driven synthetic datasets for Databricks with Spark and Faker UDFs.

Community
Advanced
ThomazRossitoThomazRossito
31

databricks-synthetic-data-gen

Generate story-driven synthetic datasets in Databricks with Spark and Faker.

Community
Advanced
leary-pokenleary-poken
1

databricks-synthetic-data-gen

Generate scalable synthetic data for Databricks workloads using Spark and Faker.

Community
Advanced