audiocraft-audio-generation

Load a pretrained model with MusicGen.get_pretrained, set generation parameters like duration and temperature, then call model.generate with a list of text descriptions. Save the returned waveform with torchaudio.save at 32kHz.

MusicGen generates music from text prompts and supports melody conditioning and stereo output at 32kHz. AudioGen generates environmental sounds and sound effects from text at 16kHz. Both are part of the AudioCraft library.

Yes, MusicGen is available in Transformers as MusicgenForConditionalGeneration with AutoProcessor. You provide text inputs and call model.generate with parameters like guidance_scale and max_new_tokens instead of AudioCraft's native API.

MusicGen-small needs about 4GB VRAM in FP32 or 2GB in FP16, medium needs about 8GB or 4GB, and large needs about 16GB or 8GB. Reduce duration or use the small model if you hit CUDA out-of-memory errors.

Silent output usually means the waveform needs normalization to a target loudness. Poor quality improves with a larger model, higher cfg_coef for stronger text adherence, lower temperature, and more specific prompts describing genre, instruments, and tempo.

MusicGen typically generates up to 30 seconds per call, with duration set via set_generation_params. For longer pieces, generate segments and concatenate them, or use audio continuation with a prompt audio input.