skills.rest
kvet
kvetCOMMUNITY
Explore all skills by this author →
GitHub

flaky-tests

Run tests sequentially multiple times and generate a flaky-test report.

24|1|Updated Oct 28, 2025
Search Tags:#reporting#reliability#sequential#ci#tests#flaky
One-click install
npx skills add https://github.com/kvet/queuert --skill flaky-tests
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: flaky-tests
Source: https://github.com/kvet/queuert/tree/main/.claude/skills/flaky-tests
Command: npx skills add https://github.com/kvet/queuert --skill flaky-tests

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps teams quickly identify flaky tests by running the test suite multiple times and analyzing intermittent failures, improving CI reliability and test confidence.

Core Features & Use Cases

  • Sequential test runs: Execute tests one at a time to avoid resource contention and accurately surface flakiness.
  • Failure pattern analysis: Detect tests that pass in some runs and fail in others, and compute failure rates.
  • Reporting & filtering: Generate a structured report with run-by-run results and optional test-name filters for targeted analysis.

Quick Start

Use this skill with common invocations to run the test suite multiple times and identify flaky tests. Examples:

  • /flaky-tests → 10 runs with pnpm test --run
  • /flaky-tests 5 → 5 runs
  • /flaky-tests 20 --filter=worker → 20 runs with filter

Frequently Asked Questions about flaky-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I identify flaky tests in my CI pipeline?▼

To identify flaky tests, run your test suite multiple times sequentially and analyze intermittent failures across runs. This approach enforces deterministic execution and aggregates results to surface unreliability in your tests.

Why does my test suite pass locally but fail intermittently in CI?▼

Intermittent CI failures often indicate flaky tests caused by resource contention or non-deterministic behavior. Running tests sequentially multiple times helps accurately surface this flakiness by isolating each test execution.

How do I run tests multiple times to check for flakiness?▼

You can run tests multiple times by specifying the number of iterations in your CLI arguments, such as requesting 5 or 20 runs. The tool executes the test suite sequentially for each run and generates a structured report with run-by-run results.

Can I filter test runs to target specific unreliable tests?▼

Yes, you can apply optional test-name filters during your test runs for targeted analysis. This allows you to execute and analyze failure patterns for specific tests rather than running the entire suite every time.

What is the best way to detect intermittent test failures locally?▼

The best way to detect intermittent test failures locally is by executing your test suite sequentially multiple times to avoid resource contention. Analyzing the failure patterns across these runs computes failure rates and produces a structured flaky-test report.

Related Skills

Explore More Software Engineering Skills→
nirholasnirholas
71

run-examples

Run EVM and Solana example integration tests against testnets and local facilitators.

Community
Advanced
mvillmowmvillmow
18

test-diff-analyzer

Analyze test run differences to identify flaky tests and consistency issues.

Community
Intermediate
AscendAscend
31

run-mindspeed-llm-test

Automate MindSpeed-LLM unit tests and generate coverage reports.

Official
Advanced
marcusgollmarcusgoll
91

parallel-execution-optimizer

Convert sequential workflows into parallel execution with dependency analysis and batching.

Community
Advanced