cross-run-consistency

Detect test non-determinism by executing the same tests N times in a single session and diffing the outputs. This approach classifies differences by root cause taxonomy, providing immediate cross-run reproducibility validation for regression suites.

Cross-run consistency checking executes critical tests multiple times within a single session to compare and diff outputs. It enforces strict mode for P0 scenarios, ensuring identical results across N/N runs without tolerance fuzzing or silent averaging before releases.

Classify flaky test root causes by capturing a baseline output, executing sequential runs with artifact collection, and applying diff logic against a predefined taxonomy. The process outputs a consistency report detailing the classified non-determinism.

Yes, you can run on-demand determinism checks across code changes and environments. By applying per-test mode resolution (strict or tolerant), the checks validate cross-run reproducibility and output a consistency report with artifacts for regression workflows.

Strict mode requires identical outputs across all N runs, typically gating P0 scenarios with zero tolerance fuzzing. Tolerant mode allows per-test resolution for non-critical tests, capturing artifacts and classifying differences without failing the entire consistency check.

Use an immediate cross-run check when you need to verify if a test agrees with itself right now, complementing historical flake tracking. It is ideal for on-demand gate checks before releases and during regression workflows to ensure strict consistency.