skills.rest
twentyhq
twentyhqOFFICIAL
Explore all skills by this author →
WebsiteGitHub

qa-scout

Tests pull requests in a real browser and verifies database and log effects.

56.0k|8.9k|Updated Dec 1, 2022
Search Tags:#QA#regression testing#playwright#Browser Automation#ci#E2E Testing#Pull request
One-click install
npx skills add https://github.com/twentyhq/twenty --skill qa-scout
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: qa-scout
Source: https://github.com/twentyhq/twenty/tree/main/.claude/skills/qa-scout
Command: npx skills add https://github.com/twentyhq/twenty --skill qa-scout

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Code review and deterministic e2e suites miss user-visible regressions and silent backend failures, such as records saving in the UI while background jobs throw errors. This Skill performs browser-based QA of a merged or labeled pull request against a running Twenty app, catching bugs that look fine on screen but break under the surface.

Core Features & Use Cases

  • Diff-Driven Scoping: Reads the PR metadata, changed files, and diff to derive 2-4 high-risk user-visible scenarios, biasing toward writes and cross-object side effects like timeline entries and notifications.
  • Real Browser Execution: Drives the app with the Playwright MCP browser, logging into a workspace and executing each scenario as a user would.
  • Log and Database Attestation: Marks log offsets before testing, greps server and worker logs for new exceptions after each flow, and confirms writes with read-only psql queries.
  • Structured Verdicts: Writes a verdict.json (PASS, INVESTIGATE, or FAIL) and a GitHub-flavored report.md posted as a PR comment, with screenshots as evidence.
  • Use Case: After a PR touching timeline activity merges to main, CI invokes this Skill to create a record in the browser, confirm the timeline row landed in Postgres, and flag a swallowed worker exception that the UI never surfaced.

Quick Start

Run the qa-scout skill against PR 12345 with the context files and output directory prepared, and have it produce a verdict and report.

Frequently Asked Questions about qa-scout

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run browser QA on a pull request automatically?▼

Invoke the qa-scout skill with the PR number plus paths to context files (pr.json, files.json, pr.diff) and an output directory. It derives user-visible scenarios from the diff, executes them in a Playwright-driven browser, and writes a verdict and report.

How to catch backend errors that the UI does not show?▼

Mark log line offsets with wc -l before testing, then tail the server and worker logs after each scenario and grep for stack traces and QueryFailedError entries. Confirm writes with read-only psql queries, such as selecting the timelineActivity row after a save.

Can qa-scout run locally instead of in CI?▼

Yes. Start the Twenty stack with yarn start or the e2e recipe, then run Claude Code with the Playwright MCP configured and invoke /qa-scout with a PR number and context paths. Local URLs and credentials come from packages/twenty-e2e-testing/.env.example.

What verdicts does the QA report produce?▼

The skill writes verdict.json with PASS, INVESTIGATE, or FAIL plus a headline and per-scenario results, and a report.md posted as a PR comment on non-PASS outcomes. FAIL means reproducible breakage or a new backend exception; INVESTIGATE covers unreproducible anomalies.

When does qa-scout skip browser testing?▼

When the diff has no user-visible surface, such as CI, docs, tooling, or type-only changes, it writes a final PASS verdict with an empty scenario list explaining why. It also stops immediately with a FAIL verdict if the app does not boot.

Related Skills

Explore More Software Engineering Skills→
D3HunterD3Hunter

prepare-pr-diff-worktree

Create an isolated git worktree for a GitHub pull request and export its diff.

Community
Advanced
awais786awais786

team-status

Aggregate team PRs, issues, and merges into a status report.

Community
Intermediate
ricablericable

qa-regression

Automate QA regression testing with reusable Playwright test suites.

Community
Advanced
cosmicstack-labscosmicstack-labs
364

e2e-testing

Write stable end-to-end tests for web apps using Playwright and Cypress.

Official
Intermediate