What problem does it solve?
HeartMuLa provides open-source tools to generate music conditioned on lyrics and tags, enabling offline, customizable music creation without relying on proprietary models. It encompasses a family of components (HeartMuLa, HeartCodec, HeartTranscriptor, HeartCLAP) to deliver end-to-end music generation, audio encoding, transcription, and alignment.
Core Features & Use Cases
- HeartMuLa: music language model (3B/7B) for generation from lyrics + tags
- HeartCodec: 12.5Hz music codec for high-fidelity audio reconstruction
- HeartTranscriptor: Whisper-based lyrics transcription
- HeartCLAP: Audio-text alignment model
Use cases include generating songs from lyrics and tags locally, building custom music generation apps, and offline workflows for creative production.
Quick Start
Run a local generation workflow using the provided assets and models to synthesize music from a lyrics file and a tags file.