skills.rest
lookatitude
lookatitudeCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

qa-flaky-test-hunter

Diagnose flaky tests and classify their causes in CI pipelines.

6|Updated Apr 24, 2026
Search Tags:#testing#qa#remediation#ci#flaky#test-suite#quarantine
One-click install
npx skills add https://github.com/lookatitude/guild --skill qa-flaky-test-hunter
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: qa-flaky-test-hunter
Source: https://github.com/lookatitude/guild/tree/main/skills/specialists/qa-flaky-test-hunter
Command: npx skills add https://github.com/lookatitude/guild --skill qa-flaky-test-hunter

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Diagnoses flaky tests and classifies their causes to prevent silent CI failures and wasted cycles.

Core Features & Use Cases

  • Per-test entries — capture test id, reproduction command, classification, and evidence.
  • Quarantine list — move flaky tests out of blocking suites with expiry and owner.
  • Fix plan — ordered remediation steps from smallest to largest impact.
  • Prevention — guidelines to reduce flakiness in fixtures and test design.
  • Metrics — before/after flake rate if telemetry is available.

Quick Start

Run the qa-flaky-test-hunter on your flaky test suite to generate a flakiness report, quarantine list, and a concrete fix plan.

Frequently Asked Questions about qa-flaky-test-hunter

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose flaky tests and identify their root causes in a CI pipeline?▼

Diagnosing flaky tests involves classifying their causes by capturing test ids, reproduction commands, and evidence within test suites. This generates per-test reports that identify why unit, integration, and end-to-end pipelines fail silently.

What is the best way to quarantine flaky tests without breaking my test suite?▼

Quarantining flaky tests moves them out of blocking CI suites using a structured list with assigned owners and expiry dates. This prevents silent failures while ensuring the tests are eventually addressed or removed.

How do I create a remediation plan for unstable integration and end-to-end tests?▼

Creating a remediation plan involves ordering fix steps from smallest to largest impact based on test classifications. The plan provides concrete prevention guidelines to reduce future flakiness in fixtures and test design.

Can I track before and after flake rates for my test suite?▼

You can track before and after flake rates if telemetry is available. Monitoring these metrics helps validate the effectiveness of your remediation plan and quarantine policies within your CI workflows.

When should I move flaky tests into quarantine instead of deleting them?▼

You should quarantine flaky tests when they cause wasted CI cycles but may contain valuable assertions. Quarantine lists with expiry policies ensure tests are temporarily removed from blocking suites while formal fix plans are scheduled.

Related Skills

Explore More Software Engineering Skills→
Imad-OuteImad-Oute
2

mh-cto-sdlc-test-health

Measure test suite flakiness, coverage alignment, and maintenance burden.

Community
Advanced
WindXRanWindXRan

test-flakiness

Analyze CI logs and JUnit XML to detect flaky tests and recommend quarantine actions.

Community
Advanced
mvillmowmvillmow
18

test-diff-analyzer

Analyze test run differences to identify flaky tests and consistency issues.

Community
Intermediate
sumilee-pcusumilee-pcu

test-flakiness

Detect non-deterministic test failures by analyzing CI logs and test artifacts.

Community
Advanced