skills.rest
wsman
wsmanCOMMUNITY
Explore all skills by this author →
GitHub

test-flakiness

Detect intermittent test failures by analyzing CI logs and historical test results.

18|5|Updated Nov 25, 2025
Search Tags:#failure analysis#flaky tests#regression suite#quarantine#junit xml#ci logs#test history
One-click install
npx skills add https://github.com/wsman/MY-DOGE-MICRO --skill test-flakiness-wsman
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/wsman/MY-DOGE-MICRO/tree/main/.agents/skills/test-flakiness
Command: npx skills add https://github.com/wsman/MY-DOGE-MICRO --skill test-flakiness-wsman

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill identifies tests that pass and fail intermittently, helping teams separate real regressions from unreliable CI noise.

Core Features & Use Cases

  • Run History Analysis: Aggregates outcomes across multiple CI runs to measure pass rates and spot inconsistent tests.
  • Root Cause Triage: Classifies likely causes such as timing, order dependence, random seeds, resource leaks, external state, floating point comparisons, and scene or prefab load races.
  • Flake Management: Recommends whether to quarantine, investigate, or monitor a test, and supports updating a regression suite quarantine list.
  • Use Case: After several pipeline runs show a test failing only sometimes, use this Skill to confirm flakiness, explain the probable cause, and decide the next remediation step.

Quick Start

Ask the skill to scan your available CI logs or test history and report any flaky tests with a recommendation to quarantine, investigate, or monitor.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find flaky tests in my CI pipeline?▼

Yes, it analyzes JUnit XML, pytest, Jest, cargo test, Go test, and Playwright pipelines. The Skill parses CI logs and test identifiers across these frameworks to detect intermittent test failures effectively.

What is the best way to triage the root cause of an intermittent test failure?▼

It recommends whether to quarantine, investigate, or monitor a test, and supports updating a regression suite quarantine list. This flake management approach separates real regressions from unreliable CI noise.

Can I use this for flaky test detection in Unity and Unreal game test suites?▼

Yes, flaky test detection applies to product and game test suites, including Godot, Unity, and Unreal pipelines. It requires multiple runs and pass-fail aggregation to validate test consistency across these environments.

How many CI runs do I need to detect flaky tests?▼

Detecting flaky tests requires multiple runs to aggregate pass-fail outcomes and measure pass rates. Analyzing historical test results across several CI pipeline executions ensures accurate identification of intermittent failures.

Related Skills

Explore More Software Engineering Skills→
HechaoYannetHechaoYannet

test-flakiness

Analyze CI logs to compute per-test pass rates and flag flaky tests.

Community
Advanced
jmaglyjmagly
175

flaky-detect

Analyze CI history and test execution patterns to identify flaky tests.

Community
Advanced
softmgsoftmg

github-workflow-automation

Automate GitHub Actions workflow generation and CI/CD pipeline management.

Official
Advanced
SPURSGOSPURSGO

cpp-code-review

Reviews C++ source code for security vulnerabilities, resource leaks, and coding standards.

Community
Advanced