Analyze JUnit XML and CI logs to classify flaky tests.
npx skills add https://github.com/nevesb/idol-agency-maker --skill test-flakiness-nevesb
Please help me install this Agent Skill. Skill: test-flakiness Source: https://github.com/nevesb/idol-agency-maker/tree/main/.claude/skills/test-flakiness Command: npx skills add https://github.com/nevesb/idol-agency-maker --skill test-flakiness-nevesb
Detect intermittent, non-deterministic test failures across CI runs and surface reliable guidance so teams stop ignoring red builds and can prioritize fixes or quarantine appropriately.
Run /test-flakiness scan to analyze available CI logs and return a flakiness report with recommendations.
High-intent search queries and answers about installing and using this skill.
Flaky test detection aggregates pass/fail histories from JUnit XML and plain CI logs across multiple runs. It computes fail-rate thresholds and classifies causes like timing issues, order dependencies, or random seeds to surface non-deterministic failures.
Non-deterministic test failures stem from timing or async races, order dependencies, random seeds, resource leaks, external state, float comparisons, or scene load races. Analyzing CI run artifacts helps classify these specific flakiness causes.
Yes, flakiness analysis supports game engines like Godot, Unity, and Unreal. It parses their CI run artifacts and JUnit XML outputs to compute per-test fail rates and classify intermittent failures.
To quarantine flaky tests, the analysis recommends quarantine actions based on computed fail-rate thresholds. After explicit approval, it appends quarantined tests to tests/regression-suite.md while preserving existing quarantine entries.
Generating a flakiness report involves analyzing multiple CI run artifacts to aggregate pass/fail histories. After explicit approval, it writes a dated report to production/qa/flakiness-report-[date].md with remediation recommendations.
When run history is insufficient, the analysis flags low-confidence findings rather than guessing. It requires adequate pass/fail artifacts to compute reliable fail-rate thresholds and accurately classify flaky tests.
polseguDetect nondeterministic failures by parsing CI logs and aggregating per-test pass/fail histories.
XGanQAQAnalyze CI test logs to detect intermittent PASS/FAIL flaky tests.
emcconnellAggregates CI pass/fail histories to identify intermittent test failures across Godot, Unity, Unreal projects.
kay6198835Analyze CI run logs to identify flaky tests and recommend quarantine actions.
