opp-repl-statistical-tests

You can detect scalar result regressions by running statistical tests that compare current simulation data against saved baselines, providing a stable alternative to brittle fingerprint checks for identifying significant deviations.

Managing simulation baselines involves using CLI commands to seed the initial baseline data, then utilizing the update_statistical_test_results Python API function to refresh stored baselines across your opp_repl configurations.

To run statistical tests on .sca files, execute the run_statistical_tests CLI command or Python API function, which compares current scalar results against the configured baselines to surface significant deviations.

Yes, statistical tests are suitable for CI workflows to surface headline changes in scalar results without exhaustively validating every metric, comparing current runs against saved baselines automatically.

Yes, you need a configured statistics_folder to run statistical tests, as it is required to store the saved baselines and manage the scalar result comparisons across your simulation configurations.

Statistical tests offer a stable alternative to brittle fingerprint checks because stochastic simulation results naturally vary, and statistical comparison against baselines identifies significant deviations rather than exact mismatches.