skills.rest
Automattic
AutomatticOFFICIAL
Explore all skills by this author →
Website𝕏@automatticengGitHub

fix-e2e-tests

Identifies failing E2E tests on wp-calypso PRs and opens fix PRs via a Playwright healing agent.

12.6k|2.0k|Updated Nov 10, 2015
Search Tags:#playwright#E2E Testing#teamcity#CI debugging#gitHub cli#Test automation#Pull request
One-click install
npx skills add https://github.com/Automattic/wp-calypso --skill fix-e2e-tests
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: fix-e2e-tests
Source: https://github.com/Automattic/wp-calypso/tree/main/.claude/skills/fix-e2e-tests
Command: npx skills add https://github.com/Automattic/wp-calypso --skill fix-e2e-tests

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

When a wp-calypso pull request fails its E2E checks, GitHub only shows a generic "TeamCity build failed" status, leaving developers to manually dig through TeamCity to find which Playwright tests broke and how to fix them. This Skill automates that entire investigation and repair loop.

Core Features & Use Cases

  • Failing test identification: Reads a PR's statusCheckRollup via the GitHub CLI, then queries the TeamCity REST API to list the exact failing Playwright test occurrences with spec paths, titles, and error reasons.
  • Automated fix generation: Creates a git worktree on the PR's HEAD, delegates the repair to a Playwright Test Healer agent, and presents the root cause and diff for user review.
  • Fix PR creation: Commits, pushes, and opens a ready-for-review PR against the original PR's branch so CI validates the repair.
  • Use Case: A developer sees the E2E matrix failing on PR #110080. They invoke the Skill with the PR number, pick the failing test from the presented table, review the Healer's diff, and get a fix PR opened against their branch automatically.

Quick Start

Ask the assistant to investigate and fix the failing E2E tests on wp-calypso PR 110080.

Frequently Asked Questions about fix-e2e-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find which E2E tests failed on a wp-calypso PR?▼

Invoke the Skill with the PR number or URL. It reads the PR's statusCheckRollup via the GitHub CLI, extracts failing E2E check build IDs, and queries the TeamCity REST API for the individual failing test occurrences with their error reasons.

How do I set up TeamCity access for E2E failure investigation?▼

Create an access token at teamcity.a8c.com under your profile, then run setup-token.sh in a separate terminal (not via Claude Code's ! prefix). The token is stored at ~/.config/teamcity-access-token with 0600 permissions.

Why does the Skill need a SOCKS5 proxy on localhost:8080?▼

TeamCity at teamcity.a8c.com is only reachable through Automattic's internal network on most workstations. The Skill autodetects connectivity and falls back to a SOCKS5 tunnel on localhost:8080 when direct access fails.

Can the fix PR target trunk instead of the original PR branch?▼

By default the fix PR targets the original PR's branch so its CI goes green directly. If the failure also exists on trunk, you can re-target the base branch from the GitHub UI after the PR is opened.

What happens if all failing E2E tests are muted in TeamCity?▼

The Skill filters out muted and currently-muted test occurrences at the jq layer. If no unmuted candidates remain, it reports that all failing tests are muted and stops without attempting a fix.

Why is the fix PR opened as ready-for-review instead of draft?▼

wp-calypso's E2E test matrix skips draft PRs, so a draft would get no CI validation. Opening as ready-for-review ensures CI runs immediately and confirms the Healer's repair works.

Related Skills

Explore More Software Engineering Skills→
vaclavbenesvaclavbenes

teamcity-cli

Automate TeamCity builds, job configurations, and pipeline integration via CLI.

Community
Advanced
kanade0404kanade0404

pr-monitor

Monitor GitHub pull requests until merged or closed, then trigger the retro workflow.

Community
Advanced
kirillshilinkirillshilin

debug-e2e-test

Diagnose and fix flaky Playwright E2E tests by classifying failures from traces.

Community
Intermediate
EuraxluoEuraxluo

webapp-testing

Automate local web app UI testing with Playwright scripts.

Community
Intermediate