SmartHome Video Anomaly Benchmark

To benchmark VLM for smart home video anomaly detection, run this skill against a local model to evaluate multi-frame video understanding across 1,203 YouTube clips covering seven categories. It automates video downloading, frame extraction, inference, and report generation.

Video anomaly detection in smart home environments identifies unusual events like wildlife intrusion, senior care incidents, security breaches, and package theft. This skill uses multi-frame sequence analysis to evaluate how well Visual-Language Models recognize these specific situations.

You can evaluate multiple VLMs for package theft detection by running the benchmark on each model. The skill generates comprehensive HTML reports with accuracy, precision, recall, F1-scores, and confusion matrices, allowing direct historical performance comparisons.

Yes, the video anomaly benchmark requires both ffmpeg and yt-dlp to automate video downloading and frame extraction. It also requires the openai dependency to execute VLM inference during the evaluation process.

The evaluation dataset includes seven smart home anomaly categories: Wildlife, Senior Care, Baby Monitoring, Pet Monitoring, Home Security, Package Delivery, and General Activity. These 1,203 YouTube clips test multi-frame video understanding capabilities.