What problem does it solve?
Enables local, open-source generation of full music tracks from structured lyrics and tag prompts, providing an alternative to closed-source models for users who need offline or self-hosted audio generation.
Core Features & Use Cases
- Model Suite: Includes HeartMuLa for lyric-conditioned music generation, HeartCodec for high-fidelity audio reconstruction, HeartTranscriptor for lyrics transcription, and HeartCLAP for audio-text alignment.
- Flexible Deployment: Supports lazy loading to reduce VRAM, multi-GPU splitting for performance, and CPU fallback for environments without NVIDIA GPUs.
- Practical Use Cases: Compose multilingual songs from prompts and lyrics, generate demos for artists, and batch-produce short songs for prototypes or creative workflows.
- Operational Notes: Requires Python 3.10, manual dependency upgrades and two source patches described in the documentation, and several gigabytes of pretrained checkpoints to be downloaded prior to generation.
Quick Start
Generate a 3-minute upbeat song from these lyrics and tags and save the output as output.mp3 using HeartMuLa with lazy_load enabled.