music-to-video

Generates beat-synced HyperFrames videos from music tracks using audio analysis and per-frame compositions.

Updated Sep 1, 2026
One-click install
npx skills add https://github.com/zamansepeti43/c-rak-agent --skill music-to-video-zamansepeti43
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: music-to-video
Source: https://github.com/zamansepeti43/c-rak-agent/tree/main/video-engine/.agents/skills/music-to-video
Command: npx skills add https://github.com/zamansepeti43/c-rak-agent --skill music-to-video-zamansepeti43

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires librosa, numpy, soundfile, and includes scripts (resource) and references (resource) components.

What problem does it solve? Turning a music track into a beat-synced video normally requires manual timeline editing, beat detection, and frame-by-frame motion design. This Skill automates that pipeline: it analyzes the audio once, lays out frames at real musical changes, plans per-frame treatments, and renders a final MP4. ## Core Features & Use Cases - Deterministic audio analysis: A single analyzer script produces an audiomap.json with beat grid, energy phases, rolls, silences, and key moments that drives all visual timing. - Orchestrated frame pipeline: A six-step workflow (setup, analyze, skeleton, plan, build, assemble, render) with user approval gates and one sub-agent per frame composition. - Template and asset treatments: Typography templates and motion primitives work with zero assets, while user-supplied images or videos are cut in on the same beat grid (beat-cut or ken-burns). - Use Case: Give the Skill a synthwave track and ask for a 30-second promo video; it analyzes the beat grid, plans kinetic typography frames, builds each as an HTML composition, and renders a synced MP4. ## Quick Start Use the music-to-video skill to turn my track bgm.mp3 into a beat-synced 1920x1080 video and render it to an MP4.

Frequently Asked Questions about music-to-video

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I make a beat-synced video from a music file?

Provide an audio file or a video to extract audio from; the skill analyzes it with analyze-beatgrid.py to produce an audiomap.json, then lays out frames, plans per-frame treatments, builds HTML compositions, and renders a final MP4 with the hyperframes CLI.

What audio analysis libraries does the beat detection use?

The analyzer uses librosa for beat tracking and onset detection, numpy for numerical processing, and soundfile for audio decoding, with ffmpeg handling format conversion. It classifies drum hits, detects rolls, energy phases, silences, and hard stops deterministically.

Can I use my own images or videos in the generated video?

Yes, user-supplied media is staged into the project with stage-assets.mjs and cut into frames on the same beat grid using beat-cut, ken-burns, or background-under-text treatments. Assets are optional since typography templates alone can produce a complete video.

Does beat-synced cutting work with calm or ambient music?

On calm music the detected beat grid is unreliable, so frames are marked phrase_flow and paced by phrases and energy envelopes instead of hard beat cuts. The pacing decision is made per frame from onset density, rolls, and energy phases.

What are the limitations of the music-to-video pipeline?

There is no narration or website capture support, and clip audio playback with BGM ducking is deferred. Frame counts stay small (about 1-6) since density is handled inside groups, and rendering requires the hyperframes CLI with headless Chrome validation.