video-understand

Extract frames and transcribe audio from video files using ffmpeg and Whisper.

Updated Sep 1, 2026
One-click install
npx skills add https://github.com/zamansepeti43/c-rak-agent --skill video-understand-zamansepeti43
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: video-understand
Source: https://github.com/zamansepeti43/c-rak-agent/tree/main/video-engine/.agents/skills/video-understand
Command: npx skills add https://github.com/zamansepeti43/c-rak-agent --skill video-understand-zamansepeti43

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires openai-whisper, and includes scripts (resource) and references (resource) components.

What problem does it solve? Understanding what a video contains normally requires watching it manually or sending it to a cloud API. This Skill analyzes video content entirely locally by extracting representative frames and transcribing the audio track, producing structured JSON output without any API keys. ## Core Features & Use Cases - Frame Extraction: Pull key frames using scene-change detection, codec keyframes, or fixed intervals, with automatic subsampling to a configurable frame limit. - Local Audio Transcription: Transcribe speech with openai-whisper (Python package or CLI), with selectable model sizes from tiny to large. - Structured JSON Output: Get video metadata, frame paths with timestamps, and transcript segments in a documented schema for downstream analysis. - Use Case: Given a screen recording of a bug report, extract scene-change frames and the narrator's transcript so an AI agent can visually inspect the frames and explain what happens in the video. ## Quick Start Analyze the video file demo.mp4 by extracting its key frames and transcribing its audio, then summarize what the video contains.

Frequently Asked Questions about video-understand

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I transcribe a video's audio locally without an API key?

Run the understand_video.py script on your video file; it extracts 16 kHz mono WAV audio with ffmpeg and transcribes it using the openai-whisper Python package or CLI. No API keys are required since Whisper runs entirely on your machine.

How to extract key frames from a video with ffmpeg?

The script supports three extraction modes: scene detection using ffmpeg's scene-change filter, keyframe mode extracting codec I-frames, and interval mode sampling frames evenly across the duration. Select a mode with the -m flag, for example -m keyframe.

What dependencies are required for local video transcription?

ffmpeg and ffprobe are required for frame extraction and audio conversion. Transcription additionally needs openai-whisper installed via pip; without it, the script still extracts frames but returns null transcript fields.

Can I process a YouTube URL directly with this video tool?

No, YouTube URLs are rejected with an error message. You must first download the video using a separate download tool, then run the script on the downloaded local file.

Why does scene detection mode extract no frames from my video?

Scene detection relies on ffmpeg's scene-change threshold of 0.3, so videos without distinct scene cuts produce no frames. The script automatically falls back to interval mode in this case, sampling frames evenly across the duration.

How do I limit the number of frames extracted from a long video?

Use the --max-frames flag to cap the output, which defaults to 20 frames. When more frames are extracted than the limit, the script subsamples evenly while keeping the first and last frames.