test-flakiness

To detect flaky tests from CI logs or JUnit XML artifacts, the skill aggregates pass/fail outcomes across multiple CI runs to flag tests with mixed results, identifying non-deterministic behavior. It analyzes plain-text logs or JUnit XML files to isolate intermittent failures.

Common root causes of non-deterministic test failures include timing or async issues, order dependency, random seeds, resource leaks, external state, floating point comparisons, and scene load races. The skill infers these drivers by inspecting static log and test patterns from your CI runs.

To manage quarantined tests, the skill reviews flagged tests in your regression suite and classifies flakiness severity to recommend whether to investigate and fix soon or quarantine immediately. It prepares updates to the quarantine section only on approval to maintain CI trust.

Yes, this test flakiness analysis works with Godot engine test result artifacts. It applies to workflows that scan Godot test result artifacts, including JUnit XML or plain-text logs, to aggregate pass/fail outcomes and classify non-deterministic test behavior.

Intermittent CI failures without code changes happen because of non-deterministic test behavior like timing races, order dependency, or resource leaks. The skill aggregates multiple CI run outcomes to confirm which tests are genuinely flaky and explains why they fail.

You should not use automated flakiness detection when you lack multiple CI run logs or existing quarantine history, as the skill relies on aggregating pass/fail outcomes across runs to infer root causes and recommend remediation. Single run failures are insufficient for pattern detection.