skills.rest
PostHog
PostHogOFFICIAL
Explore all skills by this author →
Website𝕏@PostHogGitHub

triaging-visual-review-runs

Inspects and triages PostHog Visual Review screenshot regression runs gating PR merges.

713|118|Updated Aug 11, 2020
Search Tags:#playwright#storybook#visual regression#triage#PostHog#screenshot testing#Pull request
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill triaging-visual-review-runs
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: triaging-visual-review-runs
Source: https://github.com/PostHog/posthog-foss/tree/main/products/visual_review/skills/triaging-visual-review-runs
Command: npx skills add https://github.com/PostHog/posthog-foss --skill triaging-visual-review-runs

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

When a PR is blocked by a failing visual-review GitHub status check, developers must figure out which screenshots changed, whether each diff is a real regression or a flake, and how to approve or tolerate snapshots — work that normally requires tab-hopping between GitHub and the Visual Review web UI.

Core Features & Use Cases

  • PR status lookup: Map a PR number, branch, or commit SHA to its Visual Review run state, summary counts, and deep link to the diff viewer.
  • Real-vs-flake verdicts: Combine git diff scope analysis, baseline/current PNG comparison, and per-story flake history to judge whether a snapshot diff is a genuine regression.
  • Queue triage and shipping: Aggregate runs needing review, approve or tolerate snapshots in the database, and finalize runs to commit baselines and green the merge gate — with a mandatory human confirmation gate before finalizing.
  • Use Case: A developer sees a red visual-review check on their PR and asks why it is blocked; the agent lists the changed snapshots, pulls the baseline and current screenshots, checks the story's flake history, and recommends approving the intended changes.

Quick Start

Ask the agent to check the Visual Review status of PR 1234 and explain which snapshots changed and whether the diffs look real or flaky.

Frequently Asked Questions about triaging-visual-review-runs

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check why a PR is blocked by the visual-review status check?▼

List Visual Review runs filtered by PR number, take the latest non-stale run, and inspect its summary counts. If changed or unresolved snapshots exist, list the run's snapshots to see which stories diffed and get the deep link to the diff viewer.

How do I tell if a screenshot diff is a real regression or a flake?▼

Combine three signals: check whether the PR's changed files match the story's source path, visually compare the baseline and current PNGs from their download URLs, and review the story's snapshot history for frequent prior changes indicating flakiness.

What is the difference between approving, tolerating, and finalizing a visual review run?▼

Approving marks changed snapshots as reviewed in the database, and tolerating marks a snapshot as a known benign variant; neither changes the baseline or the gate. Finalizing commits the approved baseline to the PR branch and greens the GitHub check, and requires explicit human confirmation.

Why does visual review finalization fail with a 409 error?▼

A 409 not_fully_resolved error means some changed or new snapshots are still unapproved, while 409 stale_run means a newer run superseded this one. A 409 sha_mismatch means the PR has newer commits, so CI must re-run before finalizing again.

When should I use the Visual Review web UI instead of the agent tools?▼

Use the web UI when you need to see the rendered diff image itself, since it displays the visual comparison directly. The agent tools are better for status checks, scope analysis, flake history, and triage decisions around the diff.

Related Skills

Explore More Software Engineering Skills→
githubnextgithubnext
5.1k

pr-screenshots

Capture Playwright screenshots and embed them in GitHub PR descriptions.

Official
Intermediate
lucasfcostalucasfcosta
62

pr-monitoring

Monitor a GitHub pull request until merge and required checks pass.

Community
Intermediate
silvio-lsilvio-l

code-change-review

Review code changes for regressions, boundary violations, and missing tests.

Community
Intermediate
vecearvecear

visual-regression-tester

Automate visual regression testing of UI with Playwright, Chromatic, or Percy.

Community
Advanced