skills.rest
solace-ill
solace-illCOMMUNITY
Explore all skills by this author →
GitHub

test-flakiness

Analyze CI logs and JUnit/XML results to detect flaky tests.

Updated Apr 26, 2026
Search Tags:#remediation#regression#flaky-tests#quarantine#test-results#ci-logs
One-click install
npx skills add https://github.com/solace-ill/vector-flow --skill test-flakiness-solace-ill
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/solace-ill/vector-flow/tree/main/.claude/skills/test-flakiness
Command: npx skills add https://github.com/solace-ill/vector-flow --skill test-flakiness-solace-ill

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Detect flaky tests by analyzing CI run logs and test results to identify tests that pass and fail inconsistently. This helps teams avoid chasing intermittent issues and maintain CI reliability.

Core Features & Use Cases

  • Flakiness detection: aggregates per-test pass/fail history across runs to highlight intermittently failing tests.
  • Remediation guidance: provides recommended actions (quarantine, investigate, monitor) and tracks changes in a regression suite.
  • Cross-project applicability: works with common CI setups and various game/engine ecosystems to surface flaky tests.

Quick Start

Run a scan of available CI logs to identify flaky tests and generate a remediation plan.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect flaky tests from CI logs and test results?▼

To detect flaky tests from CI logs, you can parse JUnit/XML results or plain text logs to build per-test pass/fail histories. This process aggregates run data to highlight intermittently failing tests across your CI pipeline.

What is the best way to quarantine flaky tests and guide remediation?▼

Quarantining flaky tests involves analyzing per-test histories to compute fail rates and classify flakiness levels. This classification generates actionable remediation guidance, recommending whether to quarantine, investigate, or monitor the affected tests.

Does flaky test detection work with plain text CI logs or do I need JUnit XML?▼

Flaky test detection works with both plain text CI logs and JUnit/XML formats. You can parse either format to extract test results, build per-test pass/fail histories, and compute fail rates across multiple CI runs.

Can I analyze test results across multiple CI runs and different project environments?▼

Yes, you can analyze test results across multiple CI runs and various project environments. The detection process applies cross-project to surface flaky tests by tracking inconsistencies in pass/fail histories regardless of the specific game or engine ecosystem.

How do I compute fail rates to classify flakiness levels in a regression suite?▼

To compute fail rates and classify flakiness levels, the system calculates the ratio of failures to total executions from your per-test histories. This data tracks changes in your regression suite and triggers actionable remediation guidance.

Related Skills

Explore More Software Engineering Skills→
jmaglyjmagly
175

flaky-detect

Analyze CI history and test execution patterns to identify flaky tests.

Community
Advanced
edheltzeledheltzel
2

Art

Generate Excalidraw-style diagrams and illustrations with a dark-mode palette.

Community
Advanced
sumilee-pcusumilee-pcu

test-flakiness

Detect non-deterministic test failures by analyzing CI logs and test artifacts.

Community
Advanced
polsegupolsegu

test-flakiness

Detect nondeterministic failures by parsing CI logs and aggregating per-test pass/fail histories.

Community
Intermediate