video-understand

Name: video-understand
Availability: InStock
Author: shige1014-dev

Extract frames and transcribe audio from local video files into JSON.

1|Updated Apr 11, 2026

Search Tags:#offline #video #transcription #whisper #ffmpeg #frame-extraction #scene-detection

Ready to run

npx skills add https://github.com/shige1014-dev/backup-OpenMontage --skill video-understand-shige1014-dev

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: video-understand
Source: https://github.com/shige1014-dev/backup-OpenMontage/tree/main/.agents/skills/video-understand
Command: npx skills add https://github.com/shige1014-dev/backup-OpenMontage --skill video-understand-shige1014-dev

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires ffmpeg, openai-whisper, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a local, offline way to extract representative frames and transcribe audio from video files so you can understand video content without cloud APIs or keys.

Core Features & Use Cases

Scene, keyframe, and interval extraction: Detect scene changes, pull encoder I-frames, or sample evenly across a clip.
Local transcription with Whisper: Optionally transcribe audio using openai-whisper if installed, producing segmented transcripts and full text.
Practical use cases: Rapidly inspect a downloaded clip to generate thumbnails for review, search spoken content, or create inputs for downstream visual analysis and captioning pipelines.

Quick Start

Use the video-understand skill to analyze "video.mp4" and return a JSON with absolute frame image paths, timestamps, and a transcript when available.

Frequently Asked Questions about video-understand

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract frames from a local video for offline content analysis?▼

Extract frames from local video files offline by detecting scene changes, pulling encoder I-frames, or sampling at regular intervals. The process outputs absolute JPEG frame paths and timestamps in JSON format for downstream visual inspection.

Can I transcribe video audio locally using Whisper without cloud APIs?▼

Transcribe video audio locally using openai-whisper without cloud APIs or keys. The transcription process produces segmented transcripts and full text outputs, which are included in the final JSON alongside extracted frame data.

What is the difference between scene detection, keyframe extraction, and interval sampling for video?▼

Scene detection identifies visual changes between cuts, keyframe extraction pulls encoder I-frames, and interval sampling captures frames at evenly spaced timestamps. These offline modes generate JPEG images with timestamps for content understanding.

Do I need FFmpeg installed to perform offline video transcription and frame extraction?▼

FFmpeg or ffprobe is required to extract frames and perform scene detection, while openai-whisper is an optional dependency for local audio transcription. Both dependencies support offline workflows without needing external cloud API keys.

What is the best way to prepare local video files for downstream visual inspection and captioning pipelines?▼

Prepare local video files for downstream pipelines by extracting representative frames and transcribing audio offline. This generates a JSON output containing absolute JPEG frame paths, timestamps, and transcript segments suitable for visual analysis.

Are there limitations when applying regular interval sampling and subsampling for video frame extraction?▼

Regular interval sampling and subsampling extract frames at fixed timestamps rather than semantic scene boundaries. This approach requires FFmpeg and generates JPEG frame paths, but may miss critical visual transitions if the interval is too large.