reseed-performance-baseline

Download the normalized_perf_*.json artifacts from the failed Buildkite performance jobs, review them, and pass them as source_results with the model_id, gpu_type, and an intent rationale. The workflow validates batch consistency, backs up existing history, and uploads one success=true record per accepted source after explicit confirmation.

The baseline uses the median of the last 5 successful baseline-eligible records. One or two seed records may not move an existing median immediately, three consistent records usually move it, and five effectively reset the window. A single record is enough for a first v2 calibration seed.

Legacy targets are scoped by (model_id, gpu_type), while v2 targets use an exact comparable identity of workload_id, variant_id, benchmark_version, hardware_profile_id, software_profile_id, and recipe_fingerprint. V2 records never fall back to legacy matching, and first v2 baselines require successful scheduled-main CALIBRATION_NEEDED artifacts.

Batches are rejected when any source record regresses against the batch median by more than max_intra_batch_regression (default 5%), when records target different identities, or when raw perf_*.json files are supplied instead of normalized artifacts. Provide cleaner reviewed sources or investigate the variance first.

Do not reseed when the failure might be a real code regression, when fixed thresholds in .buildkite/performance-benchmarks/tests/*.json are the actual blocker, or when there is no clear source run, commit, and rationale. Baseline history is a production signal requiring provenance.

The workflow keeps the /tmp tracking mirror, prepared staging records, source downloads, and backup with PROVENANCE.txt until you reply exactly 'cleanup reseed temp'. Nothing is deleted automatically, preserving audit artifacts in case a bad seed needs correction.