verbalized-sampling

To prevent mode collapse and increase LLM output diversity, you can verbalize a probability distribution over multiple candidate responses. This technique forces the model to evaluate varied options instead of collapsing to the single most likely output.

Verbalized sampling is a technique that prompts a large language model to verbalize a probability distribution over multiple candidate responses. This process mitigates mode collapse and increases output variation without requiring additional model training.

To generate diverse options for architecture decision records, apply verbalized sampling to your prompt. This method encourages the model to output a probability distribution across multiple architectural choices, yielding 1.6-2.1x more varied responses.

Yes, you can improve LLM output variation at inference time without additional training by using verbalized sampling. This approach prompts the model to express a probability distribution over candidate responses, increasing diversity by 1.6-2.1x.

Tasks that require high diversity benefit most from verbalized probability distributions, including requirements elicitation, hypothesis generation, creative writing, synthetic data generation, and architecture decision-making.

Yes, verbalized sampling works effectively for synthetic data generation. By verbalizing a probability distribution over candidate responses, it mitigates mode collapse and ensures varied synthetic data outputs without needing model retraining.