skills.rest
X-manist
X-manistCOMMUNITY
Explore all skills by this author →
GitHub

video-understand

Extract key frames and transcribe audio from video files using ffmpeg and Whisper.

Updated Jul 16, 2026
Search Tags:#offline#transcription#whisper#ffmpeg#video analysis#frame extraction#scene detection
One-click install
npx skills add https://github.com/X-manist/Cohmira --skill video-understand-x-manist
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: video-understand
Source: https://github.com/X-manist/Cohmira/tree/main/src/builtin-plugins/openmontage/.agents/skills/video-understand
Command: npx skills add https://github.com/X-manist/Cohmira --skill video-understand-x-manist

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires openai-whisper, and includes scripts (resource) and references (resource) components.

What problem does it solve? Understanding what a video contains normally requires watching it manually or sending it to a cloud API. This Skill analyzes video content entirely on your local machine, extracting representative frames and transcribing the audio track without any API keys or external services. ## Core Features & Use Cases - Frame Extraction: Pull key frames using scene-change detection, codec keyframes, or fixed intervals, with automatic subsampling to a configurable frame limit. - Local Audio Transcription: Convert the video's audio track to text using openai-whisper, with selectable model sizes from tiny to large. - Structured JSON Output: Receive video metadata, frame paths with timestamps, and transcript segments in a single JSON document ready for downstream analysis. - Use Case: Given a folder of product demo recordings, run the script on each file to get timestamped frames and transcripts, then visually inspect the frames and search the transcript text to catalog each video's content. ## Quick Start Analyze the video file demo.mp4 by extracting its key frames and transcribing its audio into a JSON summary.

Frequently Asked Questions about video-understand

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I transcribe a video file locally without an API key?▼

Run the understand_video.py script on your video file, which extracts the audio track with ffmpeg and transcribes it using the openai-whisper package. Install Whisper first with pip install openai-whisper, then choose a model size with the --whisper-model flag.

How to extract key frames from a video using ffmpeg?▼

The script offers three extraction modes: scene mode detects scene changes via ffmpeg's scene threshold filter, keyframe mode pulls codec I-frames, and interval mode samples frames at even time intervals. Use the -m flag to select a mode and --max-frames to cap the output count.

Does video transcription work without Whisper installed?▼

Frame extraction works with only ffmpeg and ffprobe installed, but transcription requires openai-whisper as a Python package or CLI binary. Without Whisper, the script still outputs frames and metadata, with transcript fields set to null.

Can I process a YouTube URL directly with this script?▼

No, YouTube URLs are explicitly rejected by the script. You must first download the video to a local file using a separate download tool, then run the script on the downloaded file path.

Why does scene detection return no frames for my video?▼

Scene detection fails when a video has no significant visual changes above the 0.3 threshold. The script automatically falls back to interval mode in this case, sampling frames evenly across the video duration.

Related Skills

Explore More Content & Communication Skills→
guoqiaoguoqiao

dl

Download videos and music from YouTube and Bilibili URLs into organized directories.

Community
Intermediate
video-production-buddyvideo-production-buddy
602

video-understand

Analyze local videos with ffmpeg frame extraction and Whisper transcription.

Official
Intermediate
nbiishnbiish

video-knowledge-extractor

Extracts structured markdown knowledge bases from YouTube videos using transcripts and VLM frame analysis.

Community
Intermediate
calesthiocalesthio
55.1k

video-understand

Extract key frames and transcribe audio from videos locally using ffmpeg and Whisper.

Community
Intermediate