audiocraft-audio-generation

Load a pretrained model with MusicGen.get_pretrained('facebook/musicgen-small'), set generation parameters like duration and temperature, then call model.generate with a list of text descriptions. Save the output tensor as a WAV file at 32000 Hz using torchaudio.

MusicGen generates music from text prompts and supports melody conditioning and stereo output at 32kHz. AudioGen generates sound effects and environmental audio like rain or traffic at 16kHz. Both use the same text-to-audio transformer architecture.

Yes, MusicGen is available in Transformers as MusicgenForConditionalGeneration with AutoProcessor. Use guidance_scale instead of cfg_coef and max_new_tokens instead of duration, roughly 50 tokens per second of audio.

MusicGen-small needs about 4GB VRAM in FP32 or 2GB in FP16, medium needs 8GB/4GB, and large needs 16GB/8GB. Reduce duration, use the small model, or clear the CUDA cache between generations if you hit out-of-memory errors.

Silent output usually means the audio needs normalization to a target loudness. Poor quality improves with a larger model, higher cfg_coef for stronger text adherence, lower temperature, and more specific prompts describing genre, instruments, and tempo.

Use Stable Audio for longer commercial music generation beyond MusicGen's typical 30-second limit. Use Bark for text-to-speech combined with music and sound effects. AudioCraft fits text-to-music, melody conditioning, and sound effect generation.