polym-eval-run-evaluation

To run an A/B model evaluation, you provide a prompt and specify Model A and Model B. The skill generates image and video outputs for both models, scores them pairwise, and saves the comparison report into a SQLite database.

Pairwise scoring is a mechanism where an evaluator compares outputs from two models to produce a winner, a tie, or a per-dimension breakdown with rationale. It applies to image and video generation comparisons to determine which model performs better.

You save model comparison results by writing an eval-result row into polym-eval-db, which is a SQLite database. This persistence allows you to store each pairwise scored comparison for later querying and aggregated model rankings.

Yes, you can use this skill for new visual benchmark creation. It orchestrates the full benchmark loop by generating image and video candidates for two specified models, scoring them pairwise, and persisting the results.

Yes, the skill requires invoking the appropriate generation and Gemini scoring skills to produce image and video URLs for Model A and Model B before applying the pairwise scoring and saving the results to the database.