skills.rest
CarlosQ96
CarlosQ96COMMUNITY
Explore all skills by this author →
GitHub

test-flakiness

Analyze CI logs and test result histories to detect flaky tests.

Updated Apr 9, 2026
Search Tags:#automation#flaky-tests#test-analysis#ci-logs#regression-suite
One-click install
npx skills add https://github.com/CarlosQ96/robot-crawler --skill test-flakiness
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/CarlosQ96/robot-crawler/tree/main/.agents/skills/test-flakiness
Command: npx skills add https://github.com/CarlosQ96/robot-crawler --skill test-flakiness

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Flaky tests pass intermittently without code changes, obscuring real failures and eroding confidence in CI pipelines.

Core Features & Use Cases

  • Log Analysis: Scans CI run logs or test result files to build pass/fail histories for each test.
  • Flakiness Detection: Applies statistical thresholds to flag high, moderate, and low‑risk flaky tests.
  • Cause Classification: Suggests likely origins such as timing issues, order dependencies, random seeds, resource leaks, external state, or floating‑point comparisons.
  • Actionable Recommendations: Generates quarantine entries, fixes suggestions, and optional detailed QA reports.
  • Use Case Example: During the polish phase of a game project, run this skill to surface intermittent test failures before a release, ensuring a stable test suite.

Quick Start

Run /test-flakiness scan to analyze recent CI logs and produce a flakiness report.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I identify flaky tests in my CI pipeline logs?▼

Detect flaky tests by analyzing CI logs and test result histories to build pass/fail records for each test. The tool applies statistical thresholds to flag high, moderate, and low-risk intermittent test failures across your projects.

What causes intermittent test failures without code changes?▼

Intermittent test failures are caused by timing issues, order dependencies, random seeds, resource leaks, external state, or floating-point comparisons. Analyzing CI run logs helps classify these likely origins to resolve the instability.

How do I analyze test result XML files to find intermittent failures?▼

Analyze test result XML or text outputs by scanning them to construct pass/fail histories. The tool reads these files, applies flakiness detection thresholds, and generates actionable recommendations like quarantine entries or fix suggestions.

Can I generate a flakiness report for my regression suite during code polishing?▼

Yes, you can scan recent CI logs during code polishing phases to produce a flakiness report. It surfaces intermittent test failures before a release, ensuring a stable regression suite by recommending fixes and quarantine entries.

What file permissions are needed to scan CI logs for test flakiness?▼

You need permission to read CI log files, test result XML or text outputs, and edit repository files such as tests/regression-suite.md. These permissions allow the tool to analyze histories and generate quarantine entries.

Related Skills

Explore More Software Engineering Skills→
jmaglyjmagly
175

flaky-detect

Analyze CI history and test execution patterns to identify flaky tests.

Community
Advanced
Negentropy-LabyNegentropy-Laby
3

test-flakiness

Analyze CI logs and test histories to identify intermittent failures.

Official
Advanced
Euraika-LabsEuraika-Labs
14

test-flakiness

Detect flaky tests in CI logs from GitHub Actions, Unity, Godot, or Unreal.

Official
Advanced
KingSharkGKingSharkG

project-discovery

Detect repository ecosystems, frameworks, quality gates, and monorepo shape without writing files.

Community
Intermediate