video-reader

Convert videos into timestamped keyframes and motion timelines for image-only LLM analysis.

76|3|Updated Aug 7, 2026
One-click install
npx skills add https://github.com/Job-Yang/jobbyang-ai-skills --skill video-reader-job-yang
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: video-reader
Source: https://github.com/Job-Yang/jobbyang-ai-skills/tree/main/skills/video-reader
Command: npx skills add https://github.com/Job-Yang/jobbyang-ai-skills --skill video-reader-job-yang

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires opencv-python-headless, numpy, matplotlib, openai-whisper, and includes scripts (resource) components.

What problem does it solve? LLMs can read images but cannot watch videos, making it impossible to analyze screen recordings, user complaint clips, or interaction bug footage. This Skill uses pure code (frame-differencing) to turn a video into timestamped keyframes plus a motion timeline, so the model can tell you what happened at second X without wasting tokens on still segments. ## Core Features & Use Cases - Zero-token frame selection: Frame-differencing runs on CPU via OpenCV to detect which seconds are moving, automatically folding away still segments before the model sees anything. - Coarse-to-fine workflow: Use scan for a motion timeline of the whole clip, zoom for high-density sampling of a suspicious range, grid for a nine-grid overview image, and transcribe for optional speech-to-text with timestamps. - Use Case: A teammate sends a screen recording of an App scrolling bug. The agent scans the video, reads the motion timeline to locate the suspicious 3-5 second window, zooms in for dense frames, and reports exactly when the panel stopped following the finger. ## Quick Start Ask the agent to look at this video and tell me what happens at each second and where the interaction goes wrong.

Frequently Asked Questions about video-reader

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I let an LLM analyze a video file?

Extract timestamped keyframes from the video and feed them to the model as images. This Skill automates that with OpenCV frame-differencing: it detects which seconds contain motion, samples frames only there, and returns a motion timeline plus frame paths for the model to read.

How to debug an App scrolling bug from a screen recording?

Run the scan subcommand to get a motion timeline showing which seconds are active, then use zoom on the suspicious range for high-density frames. Read the frames with their timestamps to build a second-by-second account of how the panel or gesture behaved versus expectations.

Does video frame extraction require ffmpeg installed?

No, OpenCV decodes video on its own for scan, zoom, and grid, so no system ffmpeg is needed. Only the optional transcribe subcommand requires system ffmpeg to extract the audio track for Whisper speech recognition.

Why does the script report missing dependencies after pip install?

This is a multiple-interpreter mismatch: pip installed packages under one python3 while the script runs under another, and pip --user stores packages per version. Run the script with the absolute path of the interpreter that has the packages, or use a venv.

What are the limitations of frame-based video analysis?

It cannot diagnose frame-rate or jank problems because sampling flattens inter-frame timing; use logs or Perfetto for performance issues. In re-shot videos with hands and shake it can judge motion direction but cannot pinpoint exact touch coordinates.

What to do when a platform blocks video uploads?

Rename the .mp4 to an allowed extension like .txt or compress it into a .zip, since platforms block the extension rather than the content. OpenCV decodes by file content, so the renamed file still processes normally after upload.