test-flakiness

To detect flaky tests from CI run logs, you parse logs from pipelines like GitHub Actions to build per-test histories, aggregate pass rates, and identify intermittent failures using threshold-based classification. This process quantifies overall test stability and outputs remediation guidance for your CI workflows.

Intermittent CI failures often indicate flaky tests caused by non-deterministic behavior. By analyzing test result histories and computing per-test pass rates, you can identify these intermittent failures, classify the underlying flakiness causes, and generate a report recommending either a fix or quarantine.

Yes, you can parse CI logs from Unity, Godot, Unreal, and GitHub Actions pipelines. The tool reads these varied CI formats to build per-test histories, identifies intermittent failures by quantifying stability, and provides actionable remediation guidance for your specific CI workflows.

The best way to quarantine flaky tests in a regression registry is to analyze test histories to compute pass rates and classify flakiness causes. Once identified, you can update the regression suite directly with quarantine or fix recommendations to stabilize CI results.

You should run flaky test detection during the Polish phase or after multiple CI runs have accumulated. Analyzing test results across several runs allows you to accurately compute per-test pass rates, identify intermittent failures, and update your regression registry with quarantine or fix recommendations.