skills.rest
MantisWare
MantisWareCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

synthetic-data

Generate synthetic test inputs from YAML dimension specifications into jsonl, yaml, or csv.

Updated Apr 29, 2026
Search Tags:#llm evaluation#synthetic data#generate#reproducibility#dimension-based#eval datasets
One-click install
npx skills add https://github.com/MantisWare/BizForge --skill synthetic-data-mantisware
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: synthetic-data
Source: https://github.com/MantisWare/BizForge/tree/main/library/skills/ai-patterns/synthetic-data
Command: npx skills add https://github.com/MantisWare/BizForge --skill synthetic-data-mantisware

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Generate synthetic test inputs for LLM evaluation pipelines by defining variation dimensions, enumerating combinations, and filtering for meaningful coverage.

Core Features & Use Cases

  • Define dimension spaces (topic, difficulty, format, length, etc.) and compute the combinatorial product.
  • Filter, sample, and produce labeled or unlabeled test cases for eval datasets, stress tests, and benchmarking.
  • Ensure reproducibility with seeds and provide outputs in common formats for downstream QA and model evaluation.

Quick Start

Run the generator with a dimensions.yaml file to produce 100 synthetic test cases.

Frequently Asked Questions about synthetic-data

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate synthetic data for LLM evaluation pipelines?▼

Generate synthetic data for LLM evaluation by defining variation dimensions in a YAML file, computing the combinatorial product, and filtering combinations for meaningful coverage. This process outputs structured test cases for benchmarking and stress testing.

What is dimension-based synthetic test case generation?▼

Dimension-based synthetic test case generation defines variation spaces like topic, difficulty, and format, then enumerates all combinations. This mechanism ensures diverse and comprehensive coverage for model evaluation datasets.

Can I ensure reproducibility when generating synthetic eval datasets?▼

You can ensure reproducibility when generating synthetic eval datasets by using built-in seed support. Specifying a seed guarantees that the same dimension definitions and constraints produce identical test cases across multiple runs.

What output formats are supported for synthetic test inputs?▼

Supported output formats for synthetic test inputs include JSONL, YAML, and CSV. You can produce evaluation datasets in these formats with optional labels to integrate smoothly with downstream QA and model evaluation tasks.

How do I filter synthetic test cases for meaningful coverage?▼

Filter synthetic test cases for meaningful coverage by applying optional constraints to your YAML dimensions file. This allows you to sample, limit, and refine the combinatorial product to ensure diverse and relevant stress testing inputs.

Do I need external dependencies to create dimension-based eval datasets?▼

No external dependencies are required to create dimension-based eval datasets. The generator operates independently, accepting a YAML dimensions file and optional constraints to produce structured outputs directly.

Related Skills

Explore More Data & Analytics Skills→
tiweentiween

bmad-spec

Converts intent input into structured SPEC kernels using Python scripts and YAML configurations.

Official
Advanced
vecearvecear

evaluation-harness

Automate LLM evaluation with golden datasets, rubrics, thresholds, and regression reports.

Community
Advanced
cuoreinpacecuoreinpace

llm-evaluation

Evaluate LLM applications with automated metrics, human feedback, and benchmarking.

Community
Advanced
kinnerkarmanishkinnerkarmanish

synthetic-data

Generate labeled synthetic test inputs from dimension-based combinatorial spaces.

Community
Advanced