hyperframes-media

Generate TTS voiceovers, background music, sound effects, and captions for HyperFrames video compositions.

2|1|Updated Jun 9, 2026
One-click install
npx skills add https://github.com/zester4/zilmate --skill hyperframes-media-zester4
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: hyperframes-media
Source: https://github.com/zester4/zilmate/tree/main/.agents/skills/hyperframes-media
Command: npx skills add https://github.com/zester4/zilmate --skill hyperframes-media-zester4

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires kokoro-onnx, soundfile, google-genai, transformers, torch, numpy, elevenlabs, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Producing the audio layer of a video composition — voiceover, background music, sound effects, transcription, and captions — normally requires juggling multiple TTS providers, music generators, and caption tooling with inconsistent word-timing formats. This Skill provides one shared audio engine that produces all of these assets from a single request file, with automatic provider fallback when credentials are missing. ## Core Features & Use Cases - Multi-provider TTS: Synthesize voiceovers via HeyGen (with native word timestamps), ElevenLabs, or local Kokoro-82M, selected automatically by available credentials. - BGM and SFX: Retrieve background music and sound effects from the HeyGen audio library, or fall back to local Lyria/MusicGen generation and a bundled 21-file SFX library when offline. - Transcription and captions: Generate word-level timestamps with Whisper, then author styled captions including karaoke, per-word emphasis, and pre-built caption components. - Background removal: Create transparent WebM/ProRes cutouts of subjects for compositing text and graphics behind presenters. - Use Case: Given a storyboard of narration lines, run the audio engine once to produce voice WAVs with word timings, a mood-matched music bed, and synced sound effects, then feed the word data into the caption pipeline for an animated subtitle track. ## Quick Start Ask the assistant to generate a voiceover, background music, and captions for your HyperFrames composition using the hyperframes-media audio engine.

Frequently Asked Questions about hyperframes-media

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate a voiceover with word timestamps for captions?

Use HeyGen TTS with the --words flag to get word timestamps in the same response as the audio. For ElevenLabs or Kokoro, which return no word data, run hyperframes transcribe on the generated WAV to produce the same flat word array.

What TTS providers does the audio engine support?

The engine tries HeyGen Starfish first when a credential exists, then ElevenLabs, then local Kokoro-82M as the always-available fallback. Provider selection is automatic but can be pinned with the provider flag to avoid silent drift.

Can I generate background music without a HeyGen API key?

Yes. Without a HeyGen credential the engine falls back to local generation using Google Lyria when a Gemini key is set, otherwise MusicGen. Generation runs detached, so run wait-bgm.mjs before assembling the composition.

Why does Whisper transcription translate my non-English audio?

The CLI default model small.en silently translates non-English audio into English. Always pass --model explicitly: use small.en for known English, or small with --language for other languages.

What happens when a sound effect or music track cannot be found?

A missing SFX cue or BGM match is skipped and logged as an anomaly; it never blocks the render. Voice and remaining assets still complete, and the composition simply omits the missing audio.

When is remove-background the wrong tool for isolating a subject?

The --background-output plate is hole-cut, not inpainted, so it cannot show a scene without the person standalone. For a clean plate you need an inpainting tool such as LaMa, ProPainter, or E2FGVI.