heartmula

Generate full songs from lyrics and style tags using the open-source HeartMuLa music models.

6|1|Updated May 11, 2026
One-click install
npx skills add https://github.com/yakeworld/Synthos --skill heartmula-yakeworld
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: heartmula
Source: https://github.com/yakeworld/Synthos/tree/main/skills/extended/external-automation/automation-skills/media/heartmula
Command: npx skills add https://github.com/yakeworld/Synthos --skill heartmula-yakeworld

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, torchtune, transformers, datasets, huggingface-hub.

What problem does it solve? Setting up and running the HeartMuLa open-source music generation stack involves fragile dependency pins, required source-code patches, GPU memory constraints, and strict input formatting rules. This Skill packages the complete installation, patching, and generation workflow so you can produce full songs from lyrics and tags without rediscovering every pitfall. ## Core Features & Use Cases - Guided Installation & Patching: Clone heartlib, create a Python 3.10 venv, upgrade datasets/transformers, and apply the required RoPE cache and HeartCodec loading patches. - VRAM-Aware Generation: Run the 3B model on GPUs with as little as 8GB VRAM using lazy loading, or split models across multiple GPUs. - Input Formatting & Quality Rules: Format lyrics with bracketed structural tags ([Verse], [Chorus]) and comma-separated style tags, while enforcing float32 for HeartCodec to avoid audio quality degradation. - Use Case: A user asks for an open-source Suno alternative to generate a 4-minute romantic piano song from custom lyrics on a single consumer GPU, producing a 48kHz stereo MP3 in roughly real time. ## Quick Start Generate a song from my lyrics file and the tags "piano,happy,wedding" using the HeartMuLa 3B model with lazy loading enabled.

Frequently Asked Questions about heartmula

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate music from lyrics with HeartMuLa?

Clone the heartlib repository, create a Python 3.10 virtual environment, download the HeartMuLa and HeartCodec checkpoints, then run examples/run_music_generation.py with your lyrics file, tags file, and output path. Lyrics use bracketed structural tags like [Verse] and [Chorus], and tags are comma-separated without spaces.

What GPU do I need to run HeartMuLa locally?

HeartMuLa requires a minimum of 8GB VRAM when using --lazy_load true, which peaks at about 6.2GB for the 3B model. 16GB or more is recommended for comfortable use, and you can split models across GPUs with --mula_device and --codec_device flags.

Can HeartMuLa run on CPU or macOS?

CPU mode works via --mula_device cpu but is extremely slow, taking 30-60+ minutes per song versus about 4 minutes on GPU. Triton is unavailable on macOS, so users without an NVIDIA GPU should use a cloud GPU service or the online demo at heartmula.github.io.

Why does HeartMuLa fail after installing with transformers 5.x?

The pinned dependencies conflict with newer packages, and models loaded on meta devices skip RoPE cache initialization. You must upgrade datasets and transformers, then patch modeling_heartmula.py to reinitialize Llama3ScaledRoPE and add ignore_mismatched_sizes=True to HeartCodec loading calls.

Why does HeartCodec audio quality degrade with bfloat16?

HeartCodec must run in float32 precision; using bfloat16 for the codec degrades audio reconstruction quality. Keep the default --codec_dtype float32 while bfloat16 remains fine for the HeartMuLa language model itself.