skills.rest
emcconnell
emcconnellCOMMUNITY
Explore all skills by this author →
GitHub

test-flakiness

Aggregates CI pass/fail histories to identify intermittent test failures across Godot, Unity, Unreal projects.

Updated Apr 10, 2026
Search Tags:#log-analysis#tests#quarantine#flakiness#ci-logs#test-history
One-click install
npx skills add https://github.com/emcconnell/nova-scout --skill test-flakiness-emcconnell
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/emcconnell/nova-scout/tree/main/.claude/skills/test-flakiness
Command: npx skills add https://github.com/emcconnell/nova-scout --skill test-flakiness-emcconnell

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Detects flaky tests by reading CI run logs or test result history, aggregates pass rates per test, identifies intermittent failures, and maintains a flaky test registry. Best run during Polish phase or after multiple CI runs.

Core Features & Use Cases

  • Flaky test detection: aggregate results across runs to identify tests that fail intermittently.
  • Regression suite integration: suggests quarantine or remediation steps and updates tests/regression-suite.md.
  • Multi-platform compatibility: supports test result formats from Godot, Unity, and Unreal projects.

Quick Start

Run the test-flakiness analysis against your available CI logs to surface flaky tests and suggested remediation.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I identify flaky tests from CI logs and pass/fail histories?▼

Detect flaky tests by aggregating pass/fail histories from CI logs and test results across multiple runs. Parsing JUnit XML and plain logs surfaces intermittent failures and generates a flakiness report.

Does flaky test detection work with Godot, Unity, and Unreal projects?▼

Yes, flaky test detection supports Godot, Unity, and Unreal projects by parsing test result formats from each platform. It aggregates JUnit XML and plain logs to identify intermittent failures across these engines.

What's the best way to quarantine flaky tests in a regression suite?▼

The best way to quarantine flaky tests is to apply quarantine guidance that updates tests/regression-suite.md. Aggregated pass/fail histories generate remediation suggestions to stabilize your regression suite and CI runs.

When should I run a flakiness analysis on my test results?▼

Run flakiness analysis during the Polish phase or after multiple CI runs. Aggregating pass/fail histories at these stages detects intermittent failures and maintains a flaky test registry to stabilize CI.

What test result formats can I parse to detect intermittent test failures?▼

Parse JUnit XML and plain logs to detect intermittent test failures. Aggregating these formats across multiple runs identifies flaky tests and generates a comprehensive flakiness report.

Related Skills

Explore More Software Engineering Skills→
solace-illsolace-ill

test-flakiness

Analyze CI logs and JUnit/XML results to detect flaky tests.

Community
Intermediate
jmaglyjmagly
175

flaky-detect

Analyze CI history and test execution patterns to identify flaky tests.

Community
Advanced
terraleiloaterraleiloa

income-statement

Generate categorized Wells Fargo income statements as Markdown and JSON from SerenDB data.

Community
Intermediate
Euraika-LabsEuraika-Labs
14

test-flakiness

Detect flaky tests in CI logs from GitHub Actions, Unity, Godot, or Unreal.

Official
Advanced