skills.rest
swyxio
swyxioCOMMUNITY
Explore all skills by this author →
Website𝕏@swyxGitHub

multimodal-extraction

Convert video inputs into timestamped markdown timelines with screenshots and transcripts.

154|9|Updated Mar 27, 2026
Search Tags:#whisper#video transcription#slide extraction#multimodal notes#markdown timeline#video recap#timestamped transcript
One-click install
npx skills add https://github.com/swyxio/skills --skill multimodal-extraction
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: multimodal-extraction
Source: https://github.com/swyxio/skills/tree/main/multimodal-extraction
Command: npx skills add https://github.com/swyxio/skills --skill multimodal-extraction

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires ffmpeg, yt-dlp, openai-whisper, tesseract.

What problem does it solve?

Manually creating notes from videos that sync screenshots with transcript text is extremely time-consuming, especially for long talks or presentations with dense slide content. This Skill automates the entire end-to-end workflow to produce a ready-to-use markdown document in minutes.

Core Features & Use Cases

  • End-to-end video processing: Automatically downloads videos from URLs or uses local files, extracts slide frames and key visual moments, transcribes audio, and merges everything into a single timestamped markdown file.
  • Three quality modes: Choose between fast swift mode for quick first-pass notes, context mode for OCR-enriched slide text integration, or polish mode for LLM-refined, high-quality deliverables.
  • Use cases: Perfect for generating study notes from recorded lectures, creating shareable recaps of conference talks, or building searchable archives of video content with synced visuals and text.

Quick Start

Use the multimodal-extraction skill to turn the attached video file or URL into a markdown timeline with synced screenshots and full transcript text.

Frequently Asked Questions about multimodal-extraction

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create timestamped markdown notes with synced video screenshots and transcripts?▼

Timestamped markdown notes are generated by extracting slide frames and transcribing audio, then interleaving the screenshots with text. This Skill automates the entire workflow, processing local or online videos to produce a ready-to-use markdown timeline.

Can I transcribe video audio locally without external API dependencies?▼

Yes, local transcription without external API dependencies is supported using openai-whisper. The Skill handles media download via yt-dlp and audio preprocessing through ffmpeg to execute the entire pipeline on your machine.

How do I extract text from presentation slides during video transcription?▼

Extracting text from presentation slides is handled in context mode, which applies OCR-enriched integration via tesseract. This captures visual anchor frames and extracts the embedded text to merge with the timestamped transcript.

Do I need ffmpeg and yt-dlp to extract visual anchors and transcribe online video URLs?▼

Yes, ffmpeg and yt-dlp are required dependencies to process online video URLs. ffmpeg handles audio preprocessing and frame extraction, while yt-dlp manages the video download for subsequent transcription and screenshot generation.

What is the best way to generate shareable recaps from recorded lectures and webinars?▼

Generating shareable recaps is best handled by polish mode, which uses LLM refinement for high-quality deliverables. This produces a searchable markdown archive with synced visuals and text from recorded lectures or webinars.

Why use local video processing for multimodal notes instead of manual transcription?▼

Manual note creation that syncs screenshots with transcript text is extremely time-consuming for dense presentations. Automating the extraction process reduces this workload to minutes, producing a structured timeline with visual anchors and accurate audio transcriptions.

Related Skills

Explore More Education & Research Skills→
xxihxxih
1

video-processor

Download, convert, and transcribe online videos using yt-dlp, FFmpeg, and Whisper.

Community
Intermediate
ahostbrahostbr

ls-youtube-transcript

Extract YouTube subtitles into timestamped markdown transcripts via yt-dlp.

Community
Intermediate
costiashcostiash
4

transcription-helper

Transcribe local videos and YouTube content into timestamped transcripts using gpt-4o-transcribe.

Community
Advanced
zhcndoczhcndoc

openai-whisper

Transcribe audio files locally using the Whisper CLI.

Official
Intermediate