What problem does it solve?
Automates the tedious, error-prone process of building reference C encoders, running encoding experiments across multiple sequences and QP points, and producing quantitative rate-distortion comparisons (BD-PSNR / BD-rate) so teams can objectively compare codec algorithm candidates.
Core Features & Use Cases
- Build ref C encoders from refc/ using a reproducible C11 gcc build step or existing binaries.
- Parallel encoding orchestration: local ProcessPoolExecutor or optional AWS Batch submission for large-scale evaluations.
- BD-PSNR / BD-rate calculation using VCEG-M33 polynomial fitting with 3+ QP point support and N-candidate comparisons.
- Configurable parsing and metrics: custom encoder CLI templates, regex output parsing, and opt-in SSIM/VMAF via ffmpeg.
- Artifacts & reporting: caches raw results under .rat/scratch/rd-eval/, computes per-sequence and aggregate metrics, and renders a markdown report from a template.
- Use Cases: algorithm selection during DSE, fixed-point precision impact analysis, N-way candidate ranking, and regression tracking.
Quick Start
Run codec-rd-eval with a completed HJSON test-config to build encoders, execute encodes, compute BD metrics, and generate the RD evaluation report.