e2e-qa-harness

Verify frontend changes in a real browser with Playwright checklist assertions and git scope checks.

Updated Jul 27, 2026
One-click install
npx skills add https://github.com/arayaroma/ether --skill e2e-qa-harness-arayaroma
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: e2e-qa-harness
Source: https://github.com/arayaroma/ether/tree/main/skills/e2e-qa-harness
Command: npx skills add https://github.com/arayaroma/ether --skill e2e-qa-harness-arayaroma

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires @playwright/test, and includes scripts (resource) and references (resource) components.

What problem does it solve? Verifying that a UI change actually works in a real browser is often skipped or done with unreliable open-ended visual judgment. This Skill runs a fixed, executable Playwright checklist against a running frontend app and cross-checks the actual git diff against the files claimed in tasks.md, producing paste-ready evidence for the verify phase. ## Core Features & Use Cases - Fixed checklist assertions: Generates and runs concrete Playwright tests (route loads, footer visible, form submits) selected from keyword-matched templates such as footer/nav visibility, form submission, list pagination, CRUD dialog, and dashboard/cards. - Scope verification: Compares the real git diff in the app directory against the ## Files section of tasks.md and fails on any file changed outside the claimed scope. - Structured evidence output: Prints a [x]/[ ] checklist with inline failure details plus a non-zero exit code on any failure, ready to paste verbatim into evidence.md. - Use Case: After implementing a legal footer in a Next.js app, run the harness against the local dev server to confirm the footer and its links render on the target route and that no files outside the declared change scope were modified. ## Quick Start Start the frontend dev server, then ask the agent to run the e2e QA harness with the app directory and the change's tasks.md to verify the UI change in a browser.

Frequently Asked Questions about e2e-qa-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run Playwright e2e checks for a frontend change?

Start the app's dev server, then run run-e2e.mjs with --app-dir pointing at the frontend app, --tasks-md pointing at the change's tasks.md, and optionally --base-url. The script generates a Playwright spec from a checklist template, runs it, and prints a pass/fail checklist.

How do I verify a git diff matches the files claimed in a task?

The harness reads the ## Files section of tasks.md as the claimed scope and compares it against git diff, staged changes, and untracked files in the app directory. Any changed file outside the claimed list is reported as a scope violation and fails the run.

Does Playwright need to be installed in the frontend app?

Yes. @playwright/test must be a devDependency of the target app, installed with pnpm add -D @playwright/test followed by npx playwright install chromium. The generated config and spec live inside the app directory so Node's ESM resolver can find node_modules.

Why does the Playwright JSON report show zero checklist items?

The Playwright JSON reporter nests specs one level deeper than the top-level suite when test.describe is used. A scorer reading report.suites[i].specs directly sees nothing; it must recurse through suite.suites, as checklist-scorer.mjs does.

What are the limitations of automated browser checklist verification?

The generated scenario is only a smoke check: route loads plus footer visibility for footer tasks. Anything beyond that requires editing the generated items or hand-writing a spec from the checklist templates. It also cannot judge open-ended visual quality, only concrete assertions.