Analyze CI history and test execution patterns to identify flaky tests.
npx skills add https://github.com/jmagly/ai-writing-guide --skill flaky-detect
Please help me install this Agent Skill. Skill: flaky-detect Source: https://github.com/jmagly/ai-writing-guide/tree/main/.factory/skills/flaky-detect Command: npx skills add https://github.com/jmagly/ai-writing-guide --skill flaky-detect
This Skill eliminates unreliable CI builds by automatically identifying flaky tests through CI history analysis and code pattern scanning.
When you say "find flaky tests", this Skill analyzes your CI history for inconsistent test results and scans code for flaky patterns like timing issues and async dependencies.
"Flaky Test Report
Analysis Period: Last 30 days Total Tests: 450 Flaky Tests Found: 12 (2.7%)
Critical Flaky Tests (< 50% pass rate):
Root Cause: Test creates token and checks expiry in same millisecond sometimes, different millisecond other times.
Recommended Fix: Use mocked time with vi.setSystemTime()"
High-intent search queries and answers about installing and using this skill.
Flaky test detection analyzes your CI history and test execution patterns to automatically identify non-deterministic tests. The Skill scans for common failure patterns—timing issues, async dependencies, and state leaks—then produces a report ranking tests by pass rate and severity to guide remediation.
Intermittent test failures stem from timing issues, uncontrolled async operations, shared test state, and external dependencies. Flaky test detection identifies these root causes by examining multi-run execution history and code patterns, pinpointing which tests exhibit non-deterministic behavior across pipeline runs.
Yes. Flaky test detection automates the entire process by analyzing CI history over configurable periods, applying code pattern scanning for common flake triggers, and generating actionable reports without manual log review. It identifies critical flaky tests and recommends specific fixes like mocking time or controlling async behavior.
The detection report ranks flaky tests by pass rate and impact, flagging critical failures below 50% pass rate as highest priority. Tests are categorized by root cause type—timing, async, state—so you can batch similar fixes and restore CI reliability incrementally.
Flaky test detection operates on CI execution history and code patterns independent of framework choice. It analyzes test results across runs to identify unreliable tests, then scans for framework-agnostic patterns like Date.now() calls and uncontrolled async operations to recommend fixes.
conorluddyRun XCTest and XCUITest suites and analyze failures for flaky tests.
tuistAnalyze failure patterns and apply targeted corrections to fix flaky tests.
edneyreis999Diagnose Playwright E2E test failures with browser reproduction and MCP orchestration.
asynkronDetect flaky, hanging, and crashing .NET tests with per-test timeouts.
