skills.rest
inclusionAI
inclusionAIOFFICIAL
Explore all skills by this author →
WebsiteGitHub

media_comprehension

Analyzes image, audio, and video files using ffmpeg, Whisper, and scene detection.

1.2k|126|Updated Mar 14, 2025
Search Tags:#whisper#FFmpeg#audio transcription#image analysis#scene detection#video summarization#media analysis
One-click install
npx skills add https://github.com/inclusionAI/AWorld --skill media-comprehension
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: media_comprehension
Source: https://github.com/inclusionAI/AWorld/tree/main/aworld-skills/media_comprehension
Command: npx skills add https://github.com/inclusionAI/AWorld --skill media-comprehension

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires openai-whisper, Pillow, numpy, opencv-python, moviepy, scenedetect.

What problem does it solve?

Understanding the content of media files—images, audio recordings, and videos—requires specialized tooling that general file readers cannot provide. This Skill gives an agent a structured workflow to read, transcribe, and interpret media content so users can ask questions about it directly.

Core Features & Use Cases

  • Image Analysis: Reads image files (jpg, png, gif, webp, etc.) with automatic compression to under 50KB before interpretation.
  • Audio Understanding: Extracts metadata, waveform features, and speech transcriptions from audio files using ffmpeg and Whisper.
  • Video Comprehension: Detects scenes with scenedetect, extracts midpoint frames and audio tracks, then synthesizes a combined visual and narrative summary.
  • Use Case: A user drops in a recorded meeting video and asks for a summary; the Skill extracts scene frames and audio, transcribes the speech, and produces a structured report of the narrative and key points.

Quick Start

Analyze the attached video file and summarize its scenes, narration, and overall content.

Frequently Asked Questions about media_comprehension

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I transcribe audio files with Whisper?▼

Install openai-whisper with pip, load a model such as base, and call model.transcribe on the audio file. The result includes the recognized text, detected language, and time-aligned segments for each spoken phrase.

How to extract representative frames from a video?▼

Use scenedetect with a ContentDetector to find scene boundaries, then save the midpoint frame of each scene with moviepy. This captures key narrative beats more efficiently than fixed-interval sampling.

What file formats does media analysis support?▼

Supported inputs include images (jpg, png, gif, bmp, webp, svg), audio (mp3, wav, m4a, flac, aac, ogg), and video (mp4, avi, mov, mkv, webm, flv). Documents, spreadsheets, code, and archives are explicitly out of scope.

Why must images be compressed before reading?▼

Images larger than 50KB must be compressed before reading to keep file reads within size limits. The workflow uses Pillow to reduce JPEG quality and, if needed, downscale dimensions until the output is under 50KB.

Can this analyze PDFs or spreadsheets?▼

No. The Skill is restricted to media files only and explicitly excludes documents, spreadsheets, presentations, code, archives, and configuration files. Those file types should be handled by a different document-oriented skill.

Related Skills

Explore More Data & Analytics Skills→
ZJU-REALZJU-REAL
195

lark-minutes

Search Feishu Minutes recordings and retrieve metadata, media, and transcripts via lark-cli.

Official
Advanced
tomzxtomzx
5

summarize-meeting

Convert meeting transcripts into structured summaries with topics, decisions, and action items.

Community
Intermediate
vibecodevibecode
1

exa

Perform semantic web searches and extract content via the Exa API.

Official
Advanced
juliuss1907juliuss1907

ocr-and-documents

Extract text from PDFs, scanned documents, and images using OCR.

Community
Intermediate