heartmula

Generate high-fidelity music locally by providing text-based lyrics and descriptive mood tags to open-source foundation models, bypassing restrictive proprietary cloud services. This approach requires a CUDA-enabled GPU and specific model checkpoints for audio synthesis.

Local audio synthesis requires a CUDA-enabled GPU environment alongside specific Python dependencies including torch, transformers, datasets, and torchtune. You must also acquire the necessary model checkpoints to execute the generation pipeline correctly.

Yes, you can control song structure and mood by providing specific descriptive tags and structural lyric markers like Verse and Chorus. The foundation models use these tags to condition the high-fidelity audio generation for your creative projects.

The HeartCodec model ensures professional-grade audio output by reconstructing high-fidelity audio from the generated representations. This mechanism allows the system to transform text-based lyrics and tags into full-length, high-quality audio tracks.

The best way to guide song generation is to embed structural lyric markers like Verse and Chorus directly within your text-based lyrics file. Combined with descriptive mood tags, these markers condition the foundation model to produce structurally coherent audio tracks.