skills.rest
WindXRan
WindXRanCOMMUNITY
Explore all skills by this author →
GitHub

test-flakiness

Analyze CI logs and JUnit XML to detect flaky tests and recommend quarantine actions.

Updated Apr 4, 2026
Search Tags:#root cause analysis#test reliability#regression suite#quarantine#junit xml#test flakiness#ci logs
One-click install
npx skills add https://github.com/WindXRan/y3td --skill test-flakiness-windxran
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/WindXRan/y3td/tree/main/.claude/skills/test-flakiness
Command: npx skills add https://github.com/WindXRan/y3td --skill test-flakiness-windxran

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Flaky tests intermittently fail or pass without code changes, eroding trust in CI results and masking real regressions.

Core Features & Use Cases

  • CI log and test result parsing: Detects pass/fail variability by reading JUnit-style XML or plain-text CI logs across multiple runs.
  • Flakiness classification and root-cause hints: Aggregates fail-rate history per test and maps symptoms to likely causes such as timing/async, order dependency, RNG, or resource leaks.
  • Actionable remediation and reporting: Recommends whether to quarantine immediately, investigate soon, or monitor; updates a quarantine section in the regression suite and can generate an external flakiness report with per-test guidance.

Quick Start

Run /test-flakiness scan to scan available CI logs, identify flaky tests, and produce recommendations for quarantine or fixes.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect flaky tests from CI logs and JUnit XML results?▼

Flaky tests are identified by parsing JUnit XML or plain-text CI logs across multiple runs to aggregate pass/fail outcomes per test and flag inconsistent results without code changes.

Why does my regression suite fail non-reproducibly in CI?▼

Non-reproducible red CI often stems from flaky tests with intermittent pass/fail variability; analyzing historical CI run logs and quarantine records isolates tests with inconsistent outcomes for statistical aggregation and remediation.

Can I classify flaky test root causes like timing issues or resource leaks?▼

Yes, flakiness classification maps aggregated fail-rate symptoms to likely root causes including timing or async issues, order dependency, RNG, or resource leaks to guide targeted fixes.

What is the best way to quarantine flaky tests before a release?▼

The best approach evaluates statistical pass-rate aggregation and root-cause classification to recommend immediate quarantine, further investigation, or monitoring, then updates the regression suite's quarantine section accordingly.

Does this test flakiness analysis work with plain-text CI logs or only JUnit XML?▼

Test flakiness analysis supports both JUnit-style XML and plain-text CI logs, parsing either format across multiple runs to build per-test outcome history and generate actionable flakiness reports.

Related Skills

Explore More Software Engineering Skills→
MichaelSolatiMichaelSolati

test-flakiness

Parse CI logs and JUnit XML to detect flaky tests and classify root causes.

Community
Advanced
jmaglyjmagly
175

flaky-detect

Analyze CI history and test execution patterns to identify flaky tests.

Community
Advanced
easonlaoeasonlao

test-flakiness

Analyze CI test logs to classify flaky tests and recommend quarantine actions.

Community
Intermediate
polsegupolsegu

test-flakiness

Detect nondeterministic failures by parsing CI logs and aggregating per-test pass/fail histories.

Community
Intermediate