test-flakiness

Analyze CI test logs to identify flaky tests and recommend actions.

Updated May 16, 2026

Search Tags:#flaky tests #continuous integration #test reliability #CI workflow #CI run analysis

Ready to run

npx skills add https://github.com/SeongHuni/LightNShadowSurviver --skill test-flakiness-seonghuni

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/SeongHuni/LightNShadowSurviver/tree/main/GameStudio/codex/skills/test-flakiness
Command: npx skills add https://github.com/SeongHuni/LightNShadowSurviver --skill test-flakiness-seonghuni

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill identifies and manages flaky tests in continuous integration (CI) runs, ensuring that test results are reliable and consistent.

Core Features & Use Cases

Detect Flaky Tests: Reads CI run logs or test result history to identify tests that sometimes pass and sometimes fail.
Aggregate Pass Rates: Provides pass rates per test and identifies intermittent failures.
Recommend Actions: Suggests quarantine or fix for each flaky test and maintains a flaky test registry.
Use Case: Use this Skill during the polish phase or after multiple CI runs to maintain high test quality.

Quick Start

Analyze the flakiness of your tests by running 'test-flakiness scan'.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I detect flaky tests in my CI runs?▼

To detect flaky tests in CI runs, this Skill analyzes test logs and results history to identify tests that intermittently pass and fail. It parses JUnit XML and plain text logs to classify outcomes and calculate pass rates.

How does JUnit XML log parsing classify intermittent test failures?▼

JUnit XML log parsing classifies intermittent test failures by reading CI run logs and aggregating pass rates per test. This mechanism identifies inconsistent outcomes and provides actionable recommendations for each flaky test.

What is the best way to manage a flaky test registry for continuous integration?▼

The best way to manage a flaky test registry for continuous integration is to use a Skill that recommends quarantine or fixes for each flaky test. This maintains reliability and consistency across multiple CI runs.

Can I analyze test reliability using plain text logs from CI workflows?▼

Yes, you can analyze test reliability using plain text logs from CI workflows. The Skill handles parsing of plain text logs alongside JUnit XML to aggregate pass rates and identify tests with intermittent failures.

When do I need to scan for test flakiness during the development cycle?▼

You need to scan for test flakiness during the polish phase or after multiple CI runs to maintain high test quality. Scanning ensures continuous integration processes remain reliable and consistent over time.