flake-triage

Flaky test triage starts by collecting the failure signature: frequency, first and last occurrence, environments, timing, data, retries, and recent changes. Reproduce the failure under controlled conditions when practical, then classify the primary cause as a faulty test, unstable environment, or product defect with a stated confidence level.

Compare the failure pattern against controlled reproduction runs and recent changes. If the failure reproduces only under specific timing or environment conditions, suspect the test or environment; if it reproduces consistently against real behavior, classify it as a product defect and state your confidence.

Quarantine only when you can show the lost coverage visibly, assign an owner, set a deadline, and define an exit condition. Quarantine without these controls hides risk rather than managing it.

Repeated retries are evidence about the failure, not a resolution. Retries mask the underlying instability, inflate pipeline pass rates, and delay diagnosis of whether the test, environment, or product is actually at fault.

Collect the failure signature, frequency, first and last occurrence, affected environments, timing patterns, input data, retry behavior, and recent code or infrastructure changes. This context determines whether controlled reproduction is practical and guides root cause classification.