ai-evidence-analysis

What problem does it solve?

Provide automated, AI-augmented review of captured validation evidence so teams can surface hidden failures, prioritize items to inspect, and produce repeatable, auditable findings with confidence scores.

Core Features & Use Cases

• Vision + LLM analysis: Use vision-capable models to inspect screenshots and LLMs to evaluate API and CLI outputs for errors, missing fields, and regressions.
• Per-item confidence scoring: Assign 0–100 confidence scores, verdict labels (PASS/WARN/FAIL), and structured findings with severity and remediation recommendations.
• Pipeline integration & offline-safe: Run as an optional pipeline step that writes sidecar ai-analysis JSON files, respects config/env flags to disable model calls, and never mutates original evidence.
• Use case: After an end-to-end run, analyze screenshots, API JSON responses, and CLI logs to prioritize critical failures for engineers and to provide evidence-backed verdicts for releases.

Quick Start

Analyze the captured evidence inventory and produce per-file confidence scores, structured findings, and ai-analysis sidecar files for the verdict-writer to consume.