skills.rest
cenconq25
cenconq25COMMUNITY
Explore all skills by this author →
GitHub

test-flakiness

Parse CI run histories to compute per-test pass rates and maintain a flaky-test registry.

39|6|Updated May 3, 2026
Search Tags:#testing#registry#ci-cd#ci#flaky#quarantine#test-results
One-click install
npx skills add https://github.com/cenconq25/claude-code-app-studio --skill test-flakiness-cenconq25
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-flakiness
Source: https://github.com/cenconq25/claude-code-app-studio/tree/main/.claude/skills/test-flakiness
Command: npx skills add https://github.com/cenconq25/claude-code-app-studio --skill test-flakiness-cenconq25

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Non-deterministic tests slow down delivery and erode confidence in CI. This skill detects flaky tests by analyzing CI histories, aggregates per-test pass rates, and suggests quarantine or fixes to keep CI green.

Core Features & Use Cases

  • Aggregates per-test results from CI histories to identify intermittent failures and maintain a flaky-test registry.
  • Recommends actionable steps to quarantine or fix flaky tests and minimize regression risk in pipelines.
  • Use Case: After a CI streak, run this skill to surface flaky tests, pin down likely causes, and surface remediation tasks.

Quick Start

Run the test-flakiness analysis after your CI run to identify flaky tests and update the registry.

Frequently Asked Questions about test-flakiness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I identify flaky tests from CI run histories?▼

To identify flaky tests from CI run histories, ingest pipeline execution data across the last N runs to compute per-test pass rates and surface intermittent failures. This skill parses local .test-history, gh runs, Bitrise, and CircleCI to maintain a flaky-test registry.

What is a flaky test registry and how does it stabilize CI pipelines?▼

A flaky test registry is a per-test pass-rate record aggregated from CI histories that quantifies intermittent failures to stabilize CI pipelines. By tracking pass and fail stats per test, it provides phase-aligned guidance to quarantine or fix non-deterministic tests.

Does this flaky test analysis work with Bitrise and CircleCI pipelines?▼

Yes, this flaky test analysis works with Bitrise and CircleCI pipelines, alongside local .test-history files and gh runs. It parses these history sources to compute per-test statistics including pass, fail, and duration metrics across your specified number of runs.

What's the best way to quarantine flaky tests in CI runs?▼

The best way to quarantine flaky tests in CI runs is to compute per-test pass rates from CI histories and apply phase-aligned guidance. This skill recommends actionable quarantine or remediation steps to minimize regression risk and keep pipelines green.

When should I analyze test results for intermittent failures?▼

You should analyze test results for intermittent failures after a CI streak to surface flaky tests and determine quarantine or remediation actions. Running this analysis on software projects with CI pipelines across the last N runs or days quantifies non-deterministic behavior accurately.

Related Skills

Explore More Software Engineering Skills→
drndrn
23

ci-investigate

Analyze flaky CI failures across CircleCI and GitHub Actions runs.

Community
Advanced
jmaglyjmagly
175

flaky-detect

Analyze CI history and test execution patterns to identify flaky tests.

Community
Advanced
JonathanMitchell1234JonathanMitchell1234

cicd-expert

Design CI/CD pipelines with GitHub Actions, Jenkins, and GitLab CI.

Community
Advanced
jmaglyjmagly
175

regression-metrics

Track regression rates, MTTD, MTTF, escape rate, and recurrence rate from test results.

Community
Advanced