validate-scores

Compare reference and distorted inputs across scoring backends and report pairwise numerical deltas.

3|1|Updated May 28, 2026
One-click install
npx skills add https://github.com/VMAFx/vmafx --skill validate-scores
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: validate-scores
Source: https://github.com/VMAFx/vmafx/tree/main/.claude/skills/validate-scores
Command: npx skills add https://github.com/VMAFx/vmafx --skill validate-scores

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill removes the manual effort of comparing model scores across multiple backends by automating parity checks, score aggregation, and numerical difference reporting.

Core Features & Use Cases

  • Multi-backend validation: Runs the same reference and distorted inputs through every enabled backend and collects per-feature scores.
  • Parity analysis: Computes pairwise absolute differences and ULP distances to expose floating-point drift between implementations.
  • Regression verification: Helps confirm whether a new SIMD path, GPU backend, or hot-path change preserves expected scoring behavior.
  • Use case: Use it when a backend update needs proof that score outputs remain within the accepted tolerance budget before merging.

Quick Start

Ask the assistant to validate a reference and distorted clip pair across the selected backends and report any score differences that exceed the allowed ULP threshold.

Frequently Asked Questions about validate-scores

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify backend parity for VMAF score comparison across SIMD and GPU implementations?

Backend parity for VMAF score comparison is verified by running identical reference and distorted inputs through every enabled backend, computing pairwise absolute differences and ULP distances, and applying a two-ULP pass or fail gate to detect floating-point drift.

What is ULP distance measurement used for in floating-point score validation?

ULP distance measurement in floating-point score validation quantifies the numerical drift between different backend implementations, determining whether per-frame score outputs remain within an accepted tolerance budget of two ULPs for bit-exactness verification.

How do I automate regression testing for a new SIMD path to ensure scoring behavior is preserved?

Automate regression testing for a new SIMD path by parsing per-frame JSON scores from identical inputs across backends, calculating max absolute differences, and reporting any numerical deltas that exceed the allowed ULP threshold before merging.

When do I need to perform bit-exactness verification for multiple scoring backends?

Bit-exactness verification for multiple scoring backends is needed during hot-path changes, SIMD path updates, or GPU backend modifications to prove that per-feature score outputs remain within the accepted tolerance budget without manual score aggregation.

Does backend score validation work without external dependencies for per-frame JSON parsing?

Yes, backend score validation operates with no external dependencies, handling per-frame JSON score parsing, max absolute difference calculation, and ULP distance measurement internally to report pairwise numerical deltas across enabled backends.

What is the best way to check floating-point drift between scalar and GPU backend implementations?

The best way to check floating-point drift between scalar and GPU backend implementations is to compare identical reference and distorted inputs across all enabled backends, measuring pairwise ULP distances against a strict two-ULP pass or fail gate.