ai-test-runner

The AI test runner executes your Vitest suite against the src/lib/ai/ directory, categorizes failure root causes, and outputs structured diffs with proposed fixes. It enforces deterministic behavior and strict isolation to ensure reproducible validation without external LLM calls.

Yes, you can generate a prioritized Vitest test plan by running the builder mode. It analyzes your AI source files and outputs a structured, prioritized plan for fully runnable tests, allowing you to build comprehensive AI layer validation without external LLM dependencies.

AI test suites often fail due to non-deterministic behavior from external LLM calls. This runner enforces isolation by bypassing real LLM interactions, categorizing the root causes of failures, and proposing fixes through structured diffs to guarantee safe, reproducible validation.

No, the AI test runner does not apply fixes automatically. It outputs explicit structured diffs for your review, maintaining strict safety constraints and deterministic execution while ensuring you retain full control over applying the proposed changes to your AI layer.

Automated AI layer test runners are limited to specific scopes like src/lib/ai/ and enforce strict isolation to prevent real LLM calls. They propose fixes via structured diffs but cannot execute changes automatically, requiring manual review for safe, deterministic validation.