skills.rest
VMAFx
VMAFxOFFICIAL
Explore all skills by this author →
WebsiteGitHub

run-netflix-bench

Run the Netflix benchmark suite and compare results against the committed baseline.

3|1|Updated May 28, 2026
Search Tags:#benchmark#regression-testing#performance-analysis#netflix#vmaf#backend-validation
One-click install
npx skills add https://github.com/VMAFx/vmafx --skill run-netflix-bench
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: run-netflix-bench
Source: https://github.com/VMAFx/vmafx/tree/main/.claude/skills/run-netflix-bench
Command: npx skills add https://github.com/VMAFx/vmafx --skill run-netflix-bench

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill checks whether the Netflix benchmark results produced by VMAFx match the committed baseline, so you can catch performance regressions and backend-specific drift quickly.

Core Features & Use Cases

  • Benchmark Execution: Runs the full bench_all.sh suite to generate a consolidated results file for comparison.
  • Baseline Comparison: Compares the new output against the committed netflix_benchmark_results.json and flags rows that exceed the allowed tolerance.
  • Use Case: Use it after rebuilding VMAF or changing a backend to verify that resolution, feature, and backend combinations still produce expected numbers.

Quick Start

Run the Netflix benchmark suite and compare the results against the committed baseline, reporting any rows whose relative delta exceeds the allowed tolerance.

Frequently Asked Questions about run-netflix-bench

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare VMAFx benchmark results against a committed baseline?▼

To compare VMAFx benchmark results, you can run the Netflix benchmark suite which executes testdata/bench_all.sh and reports any relative delta above 1e-6 per row against the committed netflix_benchmark_results.json baseline.

Why do my VMAF benchmark numbers drift after changing backends?▼

VMAF benchmark numbers drift when backend changes affect resolution and feature combinations. Running a baseline benchmark comparison validates these combinations after a build to quickly catch performance regressions and backend-specific drift.

Do I need vmaf and vmaf_bench binaries to run Netflix benchmark regression tests?▼

Yes, running Netflix benchmark regression tests requires both the vmaf and vmaf_bench binaries locally to execute the bench_all.sh suite and generate the consolidated results file for baseline comparison.

How does baseline benchmark comparison detect performance regressions in VMAFx?▼

Baseline benchmark comparison detects VMAFx regressions by running the full bench_all.sh suite, then flagging rows where the relative delta between new output and the committed netflix_benchmark_results.json exceeds a 1e-6 tolerance.

What is the allowed tolerance when comparing Netflix benchmark outputs?▼

The allowed tolerance for comparing Netflix benchmark outputs is a relative delta threshold above 1e-6. Any benchmark row exceeding this delta is flagged as a regression or backend-specific drift.

Can I validate specific resolution and feature combinations after rebuilding VMAF?▼

Yes, you can validate resolution, feature, and backend combinations after rebuilding VMAF by running the Netflix benchmark suite, which compares newly generated outputs against the committed baseline to verify expected numbers.

Related Skills

Explore More Software Engineering Skills→
emeraldwalkemeraldwalk

golang-coding

Automate Go project setup and cross-platform build script generation.

Official
Basic
riftzen-bitriftzen-bit

benchmark

Benchmark page, API, and build performance and output JSON baselines.

Community
Intermediate
pgschemapgschema
994

Run Tests

Run Go test suites for diff, plan, and dump workflows.

Official
Intermediate
lucas-flatwhitelucas-flatwhite
43

benchmark

Automates web performance audits to detect regressions via page timing metrics.

Community
Advanced