video-understand

Extract frames from local video files offline by detecting scene changes, pulling encoder I-frames, or sampling at regular intervals. The process outputs absolute JPEG frame paths and timestamps in JSON format for downstream visual inspection.

Transcribe video audio locally using openai-whisper without cloud APIs or keys. The transcription process produces segmented transcripts and full text outputs, which are included in the final JSON alongside extracted frame data.

Scene detection identifies visual changes between cuts, keyframe extraction pulls encoder I-frames, and interval sampling captures frames at evenly spaced timestamps. These offline modes generate JPEG images with timestamps for content understanding.

FFmpeg or ffprobe is required to extract frames and perform scene detection, while openai-whisper is an optional dependency for local audio transcription. Both dependencies support offline workflows without needing external cloud API keys.

Prepare local video files for downstream pipelines by extracting representative frames and transcribing audio offline. This generates a JSON output containing absolute JPEG frame paths, timestamps, and transcript segments suitable for visual analysis.

Regular interval sampling and subsampling extract frames at fixed timestamps rather than semantic scene boundaries. This approach requires FFmpeg and generates JPEG frame paths, but may miss critical visual transitions if the interval is too large.