skills.rest
rocklambros
rocklambrosCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

auditing-synthetic-data-utility

Audit synthetic tabular data utility via TSTR and TRTR comparisons.

2|Updated May 23, 2026
Search Tags:#synthetic-data#bootstrap-ci#tstr#trtr#correlation-gap#ctgan#tabular-utility
One-click install
npx skills add https://github.com/rocklambros/rcs --skill auditing-synthetic-data-utility
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: auditing-synthetic-data-utility
Source: https://github.com/rocklambros/rcs/tree/main/skills/ml-datasci/auditing-synthetic-data-utility
Command: npx skills add https://github.com/rocklambros/rcs --skill auditing-synthetic-data-utility

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you decide whether tabular synthetic data is actually useful for downstream modeling on real-world data, instead of relying on misleading synth-only checks that can hide joint-structure failure.

Core Features & Use Cases

  • Utility certification: Compares TSTR against the TRTR baseline so you can measure whether models trained on synthetic data still perform on held-out real data.
  • Failure diagnosis: Flags marginal collapse, correlation collapse, rare-class loss, over-smoothing, and memorization so you know why utility dropped.
  • Practical decision-making: Produces a clear use-as-real, use-with-caveats, or reject verdict for sharing, benchmarking, or research collaboration.
  • Use case: A data scientist generates CTGAN output for patient records and needs to know whether a partner can train a classifier on the synth set without losing real-world predictive performance.

Quick Start

Ask the skill to audit your synthetic tabular dataset against a held-out real test split and report the TSTR, TRTR, correlation gap, bootstrap confidence interval, and final utility verdict.

Frequently Asked Questions about auditing-synthetic-data-utility

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate synthetic tabular data utility for downstream machine learning tasks?▼

To evaluate synthetic tabular data utility, you compare TSTR performance against a TRTR baseline on held-out real data, analyzing correlation gaps and bootstrap confidence intervals to certify real-world predictive performance.

What is the difference between TSTR and TRTR comparisons in synthetic data validation?▼

TSTR (Train on Synthetic, Test on Real) measures downstream utility on real data, while TRTR (Train on Real, Test on Real) establishes the baseline performance, allowing you to quantify the predictive gap caused by using synthetic data.

How do I diagnose why my CTGAN synthetic data has low utility on real tasks?▼

You diagnose CTGAN utility loss by checking for marginal collapse, correlation collapse, rare-class loss, over-smoothing, and memorization, which reveals specific joint-structure failures hidden by synth-only checks.

Does this synthetic data utility audit work with SDV, Gretel, and Mostly AI outputs?▼

Yes, the synthetic data utility audit works with SDV, Gretel, Mostly AI, CTGAN, TVAE, and Synthpop workflows, requiring schema matching and a held-out real test split to perform TRTS comparisons and downstream scoring.

What metrics are needed to certify synthetic tabular data for downstream modeling?▼

Certifying synthetic tabular data requires schema matching, a held-out real test split, marginal fidelity checks, correlation-gap analysis, multi-model downstream scoring, bootstrap confidence intervals, and clear verdict thresholds for final approval.

When should I reject synthetic data instead of using it for real-world modeling?▼

You should reject synthetic data when TSTR performance falls significantly below the TRTR baseline, failing bootstrap confidence intervals and yielding a reject verdict instead of use-as-real or use-with-caveats based on your utility thresholds.

Related Skills

Explore More Data & Analytics Skills→
vignesh2027vignesh2027
6

synthetic-data-generator

Generate synthetic tabular, time-series, and image-like datasets with privacy controls.

Community
Advanced
mirakuimirakui

databricks-unstructured-pdf-generation

Generate synthetic PDF documents with JSON metadata for RAG evaluation.

Community
Intermediate
mullzhangmullzhang

sdv-synthetic-data

Generate synthetic data with SDV for single-table and multi-table datasets.

Community
Intermediate
florianbuetowflorianbuetow
12

linddun

Performs LINDDUN privacy threat analysis across seven categories using specialized subagents in parallel.

Community
Advanced