What problem does it solve?
Flaky tests cause development teams to ignore CI failures, masking genuine code issues and reducing trust in automated testing. This Skill solves that by automatically identifying non-deterministic tests, classifying their root causes, and providing clear remediation guidance to restore CI reliability.
Core Features & Use Cases
- CI Log Analysis: Parse GitHub Actions, Godot, Unity, or Unreal CI logs and test result files to aggregate per-test pass/fail history across multiple runs.
- Flakiness Classification: Categorize intermittent failures by root cause (timing issues, order dependency, random seeds, floating point errors, etc.) with engine-specific fix recommendations.
- Registry Maintenance: Update the project's
tests/regression-suite.md quarantine section and generate optional detailed flakiness reports for the QA team.
- Use Case: A game development team running CI for their Godot project can use this Skill after 5+ test runs to spot tests that fail 30% of the time without code changes, quarantine them to keep CI green, and get exact guidance to fix the underlying async timing issue.
Quick Start
Use the test-flakiness skill to scan all available CI logs in your project and get a list of flaky tests with their failure rates and recommended fixes.
