skills.rest
rakhnianskii9
rakhnianskii9COMMUNITY
Explore all skills by this author →
GitHub

test-flakiness

Analyze CI run logs and test histories to detect flaky tests and maintain a registry.

Updated Apr 24, 2026
Search Tags:#automation#testing#logs#quality#regression#ci#flaky
One-click install
npx skills add https://github.com/rakhnianskii9/God-Godot --skill test-flakiness-rakhnianskii9
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/rakhnianskii9/God-Godot/tree/main/.github/skills/test-flakiness
Command: npx skills add https://github.com/rakhnianskii9/God-Godot --skill test-flakiness-rakhnianskii9

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Detects non-deterministic (flaky) tests by analyzing CI logs and test histories, identifies intermittent failures, and maintains a registry to guide quarantine or fixes.

Core Features & Use Cases

  • Flaky test detection across CI runs with per-test pass rate aggregation
  • Quarantine guidance and remediation planning for flaky tests
  • Automated updates to tests/regression-suite.md and optional flakiness reports

Quick Start

Run the tool against a CI log to identify flaky tests and add them to the registry for triage.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I identify flaky tests across multiple CI runs?▼

To identify flaky tests across CI runs, analyze CI logs and test-result history to detect intermittent failures and aggregate pass rates per test. This process highlights non-deterministic tests for triage.

What is a flaky test registry and how does it help with CI quality?▼

A flaky test registry tracks non-deterministic tests identified in CI logs to maintain quality. It provides quarantine guidance and remediation planning by aggregating pass rates, helping isolate intermittent failures from blocking pipelines.

Can I scan a single CI log file to detect intermittent test failures?▼

Yes, you can analyze a single CI log file to detect intermittent test failures. The tool supports scanning individual logs or all available histories to identify non-deterministic behavior and update the registry.

How do I quarantine flaky tests in a regression suite?▼

To quarantine flaky tests in a regression suite, use the generated flakiness reports and registry to guide remediation. The tool automatically updates regression-suite files to isolate non-deterministic tests from blocking CI pipelines.

Does flaky test detection work with Godot-like CI pipelines?▼

Yes, flaky test detection is designed to apply to Godot-like CI pipelines. It analyzes test-result history across multiple runs to detect intermittent failures and maintain a registry for quarantine or fix guidance.

Related Skills

Explore More Software Engineering Skills→
MichaelSolatiMichaelSolati

test-flakiness

Parse CI logs and JUnit XML to detect flaky tests and classify root causes.

Community
Advanced
jmaglyjmagly
175

flaky-detect

Analyze CI history and test execution patterns to identify flaky tests.

Community
Advanced
cenconq25cenconq25
39

test-flakiness

Parse CI run histories to compute per-test pass rates and maintain a flaky-test registry.

Community
Advanced
solace-illsolace-ill

test-flakiness

Analyze CI logs and JUnit/XML results to detect flaky tests.

Community
Intermediate