heartmula

Generates full songs from lyrics and style tags using HeartMuLa music models.

Updated Jul 7, 2026
One-click install
npx skills add https://github.com/episvr/USTB-2026-SummerInternship --skill heartmula-episvr
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: heartmula
Source: https://github.com/episvr/USTB-2026-SummerInternship/tree/main/hermes-config/skills/media/heartmula
Command: npx skills add https://github.com/episvr/USTB-2026-SummerInternship --skill heartmula-episvr

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Generating complete songs with vocals from text requires either paid services like Suno or complex local model setups. This Skill guides you through installing, patching, and running HeartMuLa, an open-source music generation model family, so you can produce songs locally from lyrics and style tags. ## Core Features & Use Cases - Lyrics-to-Song Generation: Produce full MP3 songs (48kHz stereo) conditioned on bracketed lyrics and comma-separated style tags using the 3B/7B HeartMuLa models. - VRAM-Constrained Operation: Run on GPUs with as little as 8GB VRAM using lazy loading, or split models across multiple GPUs. - Dependency & Source Patching: Includes fixes for transformers 5.x RoPE cache issues and HeartCodec checkpoint loading mismatches. - Use Case: A musician wants a local, offline Suno alternative. They write lyrics with [Verse]/[Chorus] tags, specify tags like "piano,happy,wedding", and generate a 4-minute song in about 4 minutes on a single GPU. ## Quick Start Ask the AI to install HeartMuLa and generate a song from your lyrics file and style tags using the 3B model with lazy loading enabled.

Frequently Asked Questions about heartmula

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate a song from lyrics with HeartMuLa?▼

Run the run_music_generation.py example with --lyrics and --tags files, a model path, and --version 3B. Lyrics use bracketed structural tags like [Verse] and [Chorus]; tags are comma-separated style descriptors like piano,happy,wedding.

What GPU do I need for local AI music generation?▼

HeartMuLa requires a minimum of 8GB VRAM with lazy_load enabled, which peaks around 6.2GB for the 3B model. 16GB+ VRAM is recommended, or you can split HeartMuLa and HeartCodec across two GPUs with --mula_device and --codec_device.

Can HeartMuLa run on CPU or macOS?▼

CPU mode works via --mula_device cpu but is extremely slow, taking 30-60+ minutes per song with 12GB+ RAM. Triton is unavailable on macOS, so GPU acceleration is Linux/CUDA only; consider the online demo instead.

Why does HeartMuLa fail with newer transformers versions?▼

The pinned dependencies conflict with huggingface-hub 1.x, and transformers 5.x skips RoPE cache building during meta-device loading. Apply the documented patches: reinitialize Llama3ScaledRoPE caches and add ignore_mismatched_sizes=True to HeartCodec loading.

What are the limitations of HeartMuLa song generation?▼

Style tags may be ignored since lyrics tend to dominate conditioning (known issue #90). Using bf16 for HeartCodec degrades audio quality, RTX 5080 incompatibility has been reported, and generation runs at roughly real-time speed.