synthetic-data

You can generate synthetic test inputs by defining variation dimensions like topic, difficulty, and format. The system computes their combinatorial product, filters for relevance, and outputs labeled test cases for your eval pipelines.

Dimension-based combinatorics involves defining multiple variation dimensions such as persona, length, and edge case type. The generator enumerates all combinations across these dimensions to create a comprehensive and diverse synthetic dataset.

Yes, synthetic data generation supports both data augmentation workflows and stress testing. By computing the full combinatorial space of defined dimensions and filtering for relevance, it produces diverse inputs to robustly test LLM systems.

To ensure reproducibility in synthetic test inputs, the generation process includes specific reproducibility options. This allows you to consistently recreate the same labeled datasets from your defined variation dimensions across multiple runs.

No external dependencies are required to generate synthetic datasets. The generator operates independently to define variation dimensions, compute their combinatorial space, and output labeled test inputs for your benchmarks.

The best way to create reproducible benchmarks is using dimension-based combinatorics. By defining specific variation dimensions and applying relevance filtering, you can systematically generate labeled synthetic test inputs for robust evaluation.