test-flakiness

Name: test-flakiness
Availability: InStock
Author: dnkrow

Analyze CI logs to detect intermittent test pass/fail behavior.

1|Updated May 15, 2026

Search Tags:#quality assurance #regression suite #junit xml #test flakiness #ci logs #test quarantine #intermittent failures

Ready to run

npx skills add https://github.com/dnkrow/skill --skill test-flakiness-dnkrow

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/dnkrow/skill/tree/main/claude-global/test-flakiness
Command: npx skills add https://github.com/dnkrow/skill --skill test-flakiness-dnkrow

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It identifies tests that intermittently pass or fail without code changes, helping teams stop ignoring red CI results and masking genuine regressions.

Core Features & Use Cases

Flaky test detection from CI history: Scans CI logs or reads existing test result artifacts to build per-test pass/fail outcome histories.
Failure characterization and recommendations: Classifies likely causes such as timing/async issues, order dependency, random seed, resource leaks, external state, floating-point comparisons, and scene/prefab load races.
Actionable reporting: Updates the regression suite quarantine list and can generate a detailed flakiness report for triage and fixes.

Quick Start

Run the skill in scan mode to analyze available CI logs and update your quarantined tests list in tests/regression-suite.md.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I detect flaky tests from CI logs and JUnit XML output?▼

Flaky test detection works by parsing JUnit/XML or plain-text test artifacts across multiple CI runs to aggregate per-test pass/fail outcomes and identify intermittent failures. It scans available CI logs to build a historical pass/fail profile for each test.

What causes intermittent test failures in a regression suite?▼

Intermittent test failures are classified by likely causes including timing or async issues, order dependency, random seed variations, resource leaks, external state, floating-point comparisons, and scene or prefab load races.

How do I quarantine flaky tests to stop masking real regressions?▼

You quarantine flaky tests by running the skill in scan mode to analyze CI logs and update the quarantined tests list in tests/regression-suite.md. This produces approved quarantine updates to separate intermittent failures from genuine regressions.

Can I analyze a single CI log for test flakiness, or do I need full run history?▼

You can analyze a single CI log for test flakiness using the single-log analysis mode. However, scan mode leverages prior test-result history across multiple CI runs to more accurately identify intermittent pass/fail behavior.

What is the best way to generate a flakiness report for triage and fixes?▼

The best way to generate a flakiness report is to use the skill's detailed reporting feature after analyzing your CI logs. It produces actionable reports that characterize failure causes and provide remediation guidance for triage.

Do I need specific dependencies or components to identify flaky tests?▼

You do not need specific dependencies or components to identify flaky tests. The skill requires only that your projects produce JUnit/XML or plain-text test outputs across multiple CI runs for parsing and aggregation.