synthetic-data

You can generate synthetic training data for LLM fine-tuning by using a distilabel-based pipeline to create instruction SFT datasets and preference pairs from a seed pool. This process applies mandatory quality gates like deduplication, schema validation, and LLM-as-judge scoring to ensure the output is training-ready.

The best way to create preference pairs for DPO and RLHF is to use a multi-paradigm synthetic generation pipeline that incorporates UltraFeedback approaches. It enforces instruction-following difficulty filtering and distribution rebalancing to produce high-quality, training-ready preference datasets.

Distilabel works with synthetic data generation workflows by providing an end-to-end pipeline that enforces mandatory quality gates. It integrates deduplication, schema validation, LLM-as-judge scoring, and coverage auditing to ensure generated datasets meet high coverage and quality standards before training.

Yes, you can use synthetic data generation for agent trajectory distillation. The pipeline supports agent-trajectory synthesis using FireAct and CodeAct approaches, alongside taxonomy-driven syllabus generation, to create structured training datasets for agent models.

To prevent model collapse when training on synthetic data, you should use a strong fixed teacher model for generation and avoid training on outputs from the model being trained. The pipeline also applies anti-collapse safeguards like coverage auditing and stratified-quota sampling to manage distribution skew.

Yes, synthetic data generation requires schema validation and coverage auditing as mandatory quality gates. These steps, along with deduplication, LLM-as-judge scoring, and IFD filtering, ensure the generated datasets maintain structural integrity and high coverage before being used for training.