What problem does it solve?
Flaky tests produce intermittent pass/fail results that erode trust in CI, waste developer time, and mask real regressions; this Skill finds those nondeterministic failures and helps teams decide whether to quarantine or fix them. It reduces noise in pipelines by surfacing tests with mixed outcomes across runs, explaining likely causes, and recommending concrete remediation steps.
Core Features & Use Cases
- Log parsing and aggregation: Parse JUnit XML and plain-text CI logs to build per-test pass/fail histories across multiple runs.
- Flakiness classification: Compute fail-rate thresholds (high/moderate/low) and suggest likely causes such as timing issues, order dependency, random seeds, resource leaks, external state, or floating-point comparisons.
- Actionable recommendations and reporting: Recommend quarantine, fixes, or monitoring; optionally append quarantined tests to tests/regression-suite.md and write a full flakiness report to production/qa.
- Use case: Run during the polish phase or after multiple CI runs when developers start dismissing failures as "flaky," to produce a prioritized list of tests to quarantine or fix.
Quick Start
Run /test-flakiness scan to analyze available CI logs, detect flaky tests, and propose quarantine or fixes.