ai-evidence-analysis

Automated screenshot QA and API response validation uses vision models to inspect screenshots and LLMs to evaluate API outputs, assigning 0–100 confidence scores and structured findings with severity labels. This process writes sidecar JSON files without mutating original evidence.

AI confidence scoring for post-execution validation pipelines analyzes captured evidence using file-type detection and content inspection to assign per-item 0–100 scores, verdict labels, and remediation recommendations. It operates as an optional pipeline step that outputs sidecar analysis files.

AI evidence analysis can run offline without making model calls by respecting config and environment disable flags. This ensures the pipeline avoids model requests while still performing file-type detection and content inspection within the validation workflow.

Prioritizing critical failures from CLI output inspection is best handled by applying LLMs to evaluate logs and assign per-item 0–100 confidence scores with severity labels. This automated review produces structured findings to help engineers focus on the most critical regressions.

AI evidence review does not mutate original screenshots or API responses. The analysis process performs non-destructive content inspection and writes separate sidecar ai-analysis JSON files containing the confidence scores and structured findings.

You should not use AI models for evidence review when offline mode is active or config disable flags are set. The system respects these flags to avoid model calls, ensuring no vision or LLM requests are made during the validation pipeline.