test-flakiness

Detect flaky tests in CI pipelines by analyzing test result logs from multiple runs to aggregate pass rates, classify flakiness levels, and identify intermittent failures. This process scans logs to find non-deterministic tests that pass intermittently without code changes.

Intermittent test failures in GitHub Actions are caused by non-deterministic tests that pass or fail without code changes. Analyzing CI run logs across multiple runs aggregates pass rates and classifies flakiness levels to pinpoint these unstable tests.

Yes, you can scan Unity, Godot, or Unreal project logs for flaky tests. The analysis works directly with test result artifacts from these game engine pipelines to identify intermittent failures during the polish phase.

Quarantine flaky tests in a regression suite by analyzing CI run logs to classify flakiness levels and recommend specific actions. The process proposes quarantine, fixes, or monitoring and updates the regression suite documentation automatically.

Generate a QA report for unstable tests by aggregating pass rates across multiple CI runs and classifying flakiness levels. This creates a concise flakiness summary and an optional detailed QA report for project leads.

Analyze test flakiness before a release during the polish phase of a project. Running log analysis on multiple CI runs at this stage ensures the test suite is stable and identifies intermittent failures before the release gate.