polym-eval-score-video-gemini

What problem does it solve?

It enables automated, repeatable evaluation of AI-generated videos against a text prompt, producing structured scores and rationales instead of subjective, manual grading.

Core Features & Use Cases

• Pointwise video grading: Scores a single video across instruction-following dimensions on a 0–2 scale.
• Pairwise video comparison: Compares two videos and outputs preference judgments across visual, motion, instruction-following, and optional audio dimensions (1/0/-1).
• Backend flexibility: Uses Vertex AI (default) or ByteDance AIDP, with intelligent frame-extraction fallback when needed.
• Use Case: Quickly evaluate multiple iterations of a video-generation model to decide which output better follows a creative or instruction prompt.

Quick Start

Ask an AI to evaluate a video by running the evaluation script in pointwise mode against a prompt, producing dimension-level scores and reasoning.