skills.rest
ovisan
ovisanCOMMUNITY
Explore all skills by this author →
GitHub

ai-regression-testing

Write sandbox-mode Vitest regression tests that catch AI-introduced API bugs.

2|Updated Mar 29, 2015
Search Tags:#vitest#next.js#regression testing#api testing#ai code review#bug-check#sandbox mode
One-click install
npx skills add https://github.com/ovisan/dotfiles --skill ai-regression-testing-ovisan
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ai-regression-testing
Source: https://github.com/ovisan/dotfiles/tree/main/.agents/skills/ai-regression-testing
Command: npx skills add https://github.com/ovisan/dotfiles --skill ai-regression-testing-ovisan

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When an AI coding assistant writes code and then reviews its own work, it carries the same assumptions into both steps, creating systematic blind spots where bugs survive review. This Skill provides testing patterns that mechanically catch these AI-introduced regressions, especially sandbox/production path mismatches and incomplete SELECT clauses. ## Core Features & Use Cases - Sandbox-Mode API Testing: Set up Vitest with Next.js App Router to test API routes without a database by forcing sandbox/mock mode via environment variables. - Bug-Driven Regression Tests: Write tests only for bugs that were actually found, asserting response contracts (required fields) rather than implementation details. - Bug-Check Workflow Integration: A structured workflow that runs tests and builds before any AI code review, then adds a regression test for every fix. - Use Case: After an AI assistant fixes a missing notification_settings field in a profile API, write a regression test asserting the field exists in the response so the same bug cannot be reintroduced on the fourth attempt. ## Quick Start Ask the AI to set up sandbox-mode Vitest regression tests for the API route where a bug was just fixed, asserting all required response fields are present.

Frequently Asked Questions about ai-regression-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test Next.js API routes without a database?▼

Use sandbox mode by setting environment variables like SANDBOX_MODE=true in a Vitest setup file, then call route handlers directly with a NextRequest built by a test helper. This runs API tests in under a second with no database dependency.

How do I write regression tests for bugs found in AI-generated code?▼

Write a test that asserts the exact contract the bug violated, such as checking that a required field exists in the API response, and name it after the bug (e.g., BUG-R1 regression). Add the test immediately after finding the bug, before or alongside the fix.

What is the most common regression pattern in AI-assisted development?▼

Sandbox/production path mismatch is the most common pattern, where a fix is applied to the production code path but the sandbox mock path is forgotten. Testing that both paths return the same response shape catches this mechanically.

Should I aim for high test coverage with this approach?▼

No, the strategy explicitly avoids coverage targets and instead writes tests only where bugs were actually found. Bugs cluster in complex areas like auth and multi-path logic, so tests grow organically with each fix without wasted effort.

Why is AI self-review not enough to catch bugs?▼

When the same model writes and reviews code, it carries identical assumptions into both steps, so it repeatedly approves its own mistakes. Automated tests have no such blind spot and catch the regression on the first run.

Related Skills

Explore More Software Engineering Skills→
nazrulsoftwaredevnazrulsoftwaredev

ai-regression-testing

Automates AI-assisted regression testing for API routes and sandbox/production parity using Vitest and mock-based architecture.

Community
Advanced
GrassHopper42GrassHopper42

ios-fix

Detect, repair, and verify iOS bugs in CI/CD pipelines.

Community
Advanced
JohnRebellionJohnRebellion

ai-regression-testing

Writes sandbox-mode API regression tests to catch AI-introduced code regressions.

Community
Intermediate
SirPedrSirPedr

kcd-testing

Write and review Playwright integration tests against a live PostgreSQL database.

Community
Advanced