Analyze CI logs to detect intermittent test pass/fail behavior.
npx skills add https://github.com/dnkrow/skill --skill test-flakiness-dnkrow
Please help me install this Agent Skill. Skill: test-flakiness Source: https://github.com/dnkrow/skill/tree/main/claude-global/test-flakiness Command: npx skills add https://github.com/dnkrow/skill --skill test-flakiness-dnkrow
It identifies tests that intermittently pass or fail without code changes, helping teams stop ignoring red CI results and masking genuine regressions.
Run the skill in scan mode to analyze available CI logs and update your quarantined tests list in tests/regression-suite.md.
High-intent search queries and answers about installing and using this skill.
Flaky test detection works by parsing JUnit/XML or plain-text test artifacts across multiple CI runs to aggregate per-test pass/fail outcomes and identify intermittent failures. It scans available CI logs to build a historical pass/fail profile for each test.
Intermittent test failures are classified by likely causes including timing or async issues, order dependency, random seed variations, resource leaks, external state, floating-point comparisons, and scene or prefab load races.
You quarantine flaky tests by running the skill in scan mode to analyze CI logs and update the quarantined tests list in tests/regression-suite.md. This produces approved quarantine updates to separate intermittent failures from genuine regressions.
You can analyze a single CI log for test flakiness using the single-log analysis mode. However, scan mode leverages prior test-result history across multiple CI runs to more accurately identify intermittent pass/fail behavior.
The best way to generate a flakiness report is to use the skill's detailed reporting feature after analyzing your CI logs. It produces actionable reports that characterize failure causes and provide remediation guidance for triage.
You do not need specific dependencies or components to identify flaky tests. The skill requires only that your projects produce JUnit/XML or plain-text test outputs across multiple CI runs for parsing and aggregation.
jmaglyAnalyze CI history and test execution patterns to identify flaky tests.
polseguDetect nondeterministic failures by parsing CI logs and aggregating per-test pass/fail histories.
scomofoAnalyze CI logs to detect flaky tests and classify root causes.
sumilee-pcuDetect non-deterministic test failures by analyzing CI logs and test artifacts.
