seed-flaky-test-detector

The best way to detect flaky tests is by computing per-test pass rates across recent CI runs to systematically measure stability. This method distinguishes intermittent flaky tests from consistently failing ones and generates a ranked quarantine list with severity and issue links.

Yes, flaky test detection supports CI environments that emit per-test outcomes, such as GitHub Actions, JUnit XML, and test logs. It parses these results to compute pass rates and classify the stability of each test across recent runs.

You can quarantine unstable tests by computing per-test pass rates to classify the worst offenders and generate a ranked quarantine list. This list includes severity rankings and links quarantined tests to tracking issues for actionable remediation.

Flaky tests are distinguished from consistently failing tests by computing per-test pass rates over recent runs. Flaky tests exhibit intermittent failures and unstable pass rates, whereas consistently failing tests yield continuous failures across the analyzed CI suite.

You should quarantine a flaky test when its computed pass rate over recent CI runs indicates intermittent instability rather than a consistent regression. Quarantining isolates the unstable test while linking it to tracking issues for further investigation and remediation.