skills.rest
JunnnnyWon
JunnnnyWonCOMMUNITY
Explore all skills by this author →
GitHub

test-flakiness

Analyzes CI run logs and test results to identify flaky tests and produce a regression suite report.

Updated Aug 27, 2026
Search Tags:#diagnosis#flaky-tests#quarantine#test-results#ci-logs#regression-suite
One-click install
npx skills add https://github.com/JunnnnyWon/Codex-Game-Studios --skill test-flakiness-junnnnywon
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/JunnnnyWon/Codex-Game-Studios/tree/main/.claude/skills/test-flakiness
Command: npx skills add https://github.com/JunnnnyWon/Codex-Game-Studios --skill test-flakiness-junnnnywon

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Flaky tests create unreliable CI results by intermittently passing or failing despite no code changes, which leads to wasted debugging time and delayed releases.

Core Features & Use Cases

  • Analysis of CI logs: Parse JUnit XML and plain text CI outputs to assemble per-test result histories.
  • Flakiness detection: Identify tests with pass/fail variance across runs and quantify fail rates.
  • Root-cause classification: Suggest likely causes such as timing issues, order dependencies, or resource leaks.
  • Remediation guidance: Propose quarantine or fixes and update the regression suite with clear instructions.
  • Reporting outputs: Produce an up-to-date flaky-test registry and optional failure reports for stakeholders.

Quick Start

Run the tester on a local CI log or pass a path to the CI results to start analyzing flaky tests.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I identify flaky tests from CI logs?▼

You can identify flaky tests by parsing JUnit XML or plain-text CI outputs to assemble per-test result histories and detect pass/fail variance across multiple runs.

What causes flaky tests in a regression suite?▼

Flaky tests in a regression suite are typically caused by timing issues, order dependencies, or resource leaks. Analyzing CI run logs helps classify these root causes for targeted remediation.

How do I quarantine flaky tests after detecting them?▼

To quarantine flaky tests, analyze CI logs to detect pass/fail variance, then apply remediation guidance to update the regression suite with clear quarantine instructions and structured classifications.

Does flaky test detection work with plain-text CI logs or only JUnit XML?▼

Flaky test detection works with both JUnit XML and plain-text CI logs. The analysis parses either format to assemble per-test result histories and quantify failure rates across runs.

Can I scan a repository CI folder to diagnose test flakiness?▼

Yes, you can scan a repository CI folder to diagnose test flakiness by passing a path to the CI results. The analysis produces a flaky-test registry and quarantine guidance for the identified tests.

What is the best way to generate a flakiness report for stakeholders?▼

The best way to generate a flakiness report for stakeholders is to analyze CI run logs and test results to produce an updated quarantine section, structured classifications, and optional failure reports.

Related Skills

Explore More Software Engineering Skills→
solace-illsolace-ill

test-flakiness

Analyze CI logs and JUnit/XML results to detect flaky tests.

Community
Intermediate
emcconnellemcconnell

test-flakiness

Aggregates CI pass/fail histories to identify intermittent test failures across Godot, Unity, Unreal projects.

Community
Intermediate
matteocervellimatteocervelli
25

test-executor

Execute test suites with parallelization and JUnit, HTML, and JSON reports.

Community
Intermediate
kay6198835kay6198835

test-flakiness

Analyze CI run logs to identify flaky tests and recommend quarantine actions.

Community
Advanced