skills.rest
dyad-sh
dyad-shOFFICIAL
Explore all skills by this author →
WebsiteGitHub

dyad:deflake-e2e-recent-commits

Detects and fixes flaky Playwright E2E tests from recent CI runs and PRs.

21.4k|2.6k|Updated Apr 11, 2025
Search Tags:#playwright#ci#E2E Testing#GitHub actions#Flaky tests#Test automation#deflake
One-click install
npx skills add https://github.com/dyad-sh/dyad --skill dyad-deflake-e2e-recent-commits
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: dyad:deflake-e2e-recent-commits
Source: https://github.com/dyad-sh/dyad/tree/main/.claude/skills/deflake-e2e-recent-commits
Command: npx skills add https://github.com/dyad-sh/dyad --skill dyad-deflake-e2e-recent-commits

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Flaky end-to-end tests erode trust in CI pipelines and waste developer time on false failures. This Skill automatically finds flaky Playwright tests reported in recent main-branch CI runs and PR comments, reproduces them locally, applies fixes, and verifies stability.

Core Features & Use Cases

  • Automated flaky test discovery: Scans recent CI workflow runs on main and Playwright report comments on PRs by wwwillchen, keppo-bot, and dyad-assistant to collect flaky tests.
  • Frequency-ranked deduplication: Aggregates and ranks flaky tests by occurrence count, grouped by spec file, so the worst offenders are fixed first.
  • Reproduce, fix, and verify loop: Runs each spec 10 times with retries disabled, applies Playwright best-practice fixes (explicit waits, stable selectors, snapshot updates), and re-verifies before opening a PR.
  • Use Case: After noticing intermittent CI failures on main, run this Skill to gather all recently flaky E2E tests, deflake the top offenders, and push a fix PR automatically.

Quick Start

Ask the assistant to deflake E2E tests from the last 10 commits, optionally passing a different commit count as an argument.

Frequently Asked Questions about dyad:deflake-e2e-recent-commits

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find flaky Playwright tests from GitHub Actions CI runs?▼

List completed CI workflow runs on main with the GitHub CLI, download the html-report artifact for each run, and parse results.json for tests that passed only after a failed, timedOut, or interrupted attempt. This Skill automates that entire process.

How do I reproduce a flaky Playwright test locally?▼

Run the spec file repeatedly with retries disabled: PLAYWRIGHT_RETRIES=0 npm run e2e -- e2e-tests/<spec_file> --repeat-each=10. Disabling retries is essential because CI defaults to 2 retries, which masks flakiness.

What are the best practices for fixing flaky Playwright tests?▼

Use await expect(locator).toBeVisible() before interactions, wait for networkidle on network-dependent tests, prefer stable selectors like data-testid or role, and update snapshot baselines when needed. Avoid changing application code; only modify test files.

Why does my flaky test pass locally but fail in CI?▼

CI environments set CI=true, which enables Playwright's default 2 retries, so flaky tests appear to pass. Timing differences and shared state also contribute. Reproduce with PLAYWRIGHT_RETRIES=0 and ensure proper test isolation.

What happens if a flaky test cannot be fixed?▼

The Skill limits fixing to 2 attempts per spec file. If the test still fails, it reverts any changes to that spec file, moves on to the next one, and reports the unfixed tests with reasons in the final summary.

Related Skills

Explore More Software Engineering Skills→
benzwickbenzwick

download-ci-artifacts

Download GitHub Actions CI artifacts using gh CLI commands.

Community
Intermediate
sapiencexyzsapiencexyz
21

push-pr

Push a Git branch to a remote repository and create a GitHub Pull Request.

Official
Intermediate
optaveoptave
86

test-health

Analyze vitest outputs and coverage data to produce prioritized test health reports.

Official
Intermediate
OM-bit-hubOM-bit-hub

ci-test-pipeline

Automate CI/CD pipeline setup with parallel test runs and coverage reporting.

Community
Advanced