polym-eval-score-video-gemini

To evaluate AI-generated videos against a text prompt, you can use automated video scoring to produce structured pointwise or pairwise quality judgments. This generates dimension-level scores and rationales, replacing subjective manual grading with repeatable quality assessments.

Pointwise video evaluation scores a single video across instruction-following dimensions on a 0–2 scale. Pairwise video evaluation compares two videos and outputs preference judgments across visual, motion, and instruction-following dimensions using a 1/0/-1 scale.

Yes, you can use Vertex AI via the google-genai client or ByteDance AIDP via an OpenAI-compatible API for video prompt scoring. The system defaults to Vertex AI and supports intelligent frame-extraction fallback when needed.

Pairwise comparison between video generation iterations ranks two videos by outputting preference judgments across visual, motion, instruction-following, and optional audio dimensions. This helps quickly evaluate which output better follows a creative prompt.

Yes, the video evaluation script supports optional audio-aware assessment for A/V content. During pairwise comparison, it can include audio dimensions alongside visual and motion criteria to output comprehensive preference judgments.

Structured video quality judgments are output in a strict JSON format containing dimension-level scores and reasoning. This ensures automated evaluation results are machine-readable and aligned to your defined instruction-following dimensions.