regression-test-runner

Automated regression testing diffs current test run results against a stored baseline to identify new failures and enforces a P0 pass-rate gate. For PRs, it supports a smoke mode to run the relevant test subset quickly.

Regression baseline diffing compares current test results to a stored regression-baseline.json file. It classifies outcomes as passing, new-failure, still-failing, fixed, flaky, or skipped to highlight software health changes.

Yes, the runner supports manual incremental modes alongside pr/push smoke and release full scopes. This scope-driven execution selects the relevant test subset and controls baseline updates based on the chosen scenario.

Diffing test results generates a human-readable regression-report.md, per-run JSON artifacts, and a downloadable classified.json. These outputs record per-test classifications for downstream CI engines to consume.

Yes, the runner relies on repo configuration via repo-fingerprint.json and a baseline file named regression-baseline.json. These files establish the required context to enforce the strict P0 pass-rate gate before updating the baseline.

You should not update the regression baseline when P0 tests fail. The runner enforces a strict P0 pass-rate gate, ensuring the baseline is only updated when all critical P0 tests pass to prevent masking regressions.