train-visual-qa

Automated visual QA testing for mobile UI screenshots works by running analysis prompts against labeled test cases to detect layout and spacing defects. The system collects outputs, scores them against ground truth, and applies an 80% pass threshold with zero flagged false positives.

Evaluating image analysis prompts involves running them against a library of captured test cases and computing FOUND/MISSED and AVOIDED/FLAGGED metrics. Iterative prompt tuning applies structural changes and re-runs the full suite to detect regressions until results stabilize or iteration limits are reached.

Setting up an automated test harness for UI testing requires captured test cases with expected signals, the visual-qa prompt content, and an ask_screen execution environment to run analyses and compute scoring metrics.

Visual QA prompts miss spacing defects after ad-hoc edits because untracked changes introduce regressions. Automated evaluation against labeled test cases prevents this by verifying defect detection accuracy and ensuring consistent results across all captured mobile screenshots.

The visual-qa trainer is specifically designed for mobile UI screenshot QA workflows and automated pipelines that require consistent detection of spacing and layout issues. It applies deterministic scoring to verify defect detection accuracy across many captured test cases.

Pass criteria for visual QA prompt evaluation require an 80%+ pass threshold for detecting signals and zero flagged known false positives. The system compares outputs to ground truth to mark FOUND/MISSED signals and AVOIDED/FLAGGED false positives.