noob-visual-rca

Visual diff failures are classified by examining baseline, current, and diff images to map each failure to categories like visual_regression, intentional_change, flaky_render, env_issue, threshold_issue, or unknown. This streamlines debugging by pinpointing exact root causes.

The best way to triage visual regression test failures is running a post-verification root-cause analysis that compares baseline and current images, categorizing diffs and recommending remediation actions like fix_app, update_baseline, adjust_threshold, retry, or fix_env.

During bug triage, the classification process determines if a visual diff is an intentional_change or visual_regression. An intentional_change suggests updating the baseline image, while a visual_regression recommends fixing the application code.

Yes, you can automatically file issues for visual regressions. The classification process provides an optional procedure to file a tester issue specifically for failures categorized as visual regressions, streamlining the bug reporting workflow.

Visual tests often fail due to flaky_render or env_issue categories rather than real regressions. Root-cause analysis identifies these specific failure types and recommends actions like retrying the test or fixing the environment instead of modifying application code.

No, visual root-cause analysis requires a completed verification run. It specifically applies to post-verification visual test outputs from the noob-visual and noob-visual-pool workflows to diagnose each failed comparison and guide remediation actions.