test-rot

Diagnose red, flaky, or untrustworthy tests through an ordered root-cause checklist.

Updated Aug 22, 2026
One-click install
npx skills add https://github.com/Bascht74/videopodcast-magic --skill test-rot-bascht74
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: test-rot
Source: https://github.com/Bascht74/videopodcast-magic/tree/main/.claude/skills/test-rot
Command: npx skills add https://github.com/Bascht74/videopodcast-magic --skill test-rot-bascht74

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When a test in tests/ goes red, wobbles between runs, or stays green without proving anything, it is easy to blame the program first and waste hours. This Skill enforces a cheap-to-expensive diagnostic order — environment, snapshot, contention, deadlines, stand-ins, the test itself, and only then the program — so the true cause is found before any code is changed. ## Core Features & Use Cases - Ordered root-cause elimination: Rules out language/locale mismatches (LANGUAGE=en via run.sh), stale VPM_SCRIPT snapshots, and parallel-run contention before any source reading. - Flakiness and deadline analysis: Uses wobbly.sh reports and the unsteady line to separate contention from real faults, and flags tests waiting on wall-clock deadlines instead of conditions. - False-green detection: Lists the signatures of tests that pass without checking — missing check calls, silent sys.exit(0), timer-chain escape branches, and checks that can never go red. - Use Case: A test is red on the CI builder but green locally. Following the Skill, you discover the builder is ~9x slower and the test died on a 30-second deadline — so you fix the waiting condition instead of raising the timeout. ## Quick Start Ask the AI to take apart the failing test using the test-rot checklist and identify whether the program, the test, or the stand-in is at fault.

Frequently Asked Questions about test-rot

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I debug a test that fails on CI but passes locally?▼

Check whether the failure message names a deadline, since CI builders are often many times slower and tight timeouts fail there first. Reproduce through the same run wrapper (run.sh) so locale and environment variables match, and verify the fix on the builder rather than raising the timeout.

How to tell if a flaky test is caused by parallel execution?▼

A test that is red beside others but green alone indicates contention, not a program fault. Run the suite with WORKERS=1 or repeat the single test in isolation, and consult the wobbly.sh report which tracks unsteady tests across runs and machines.

Why does my test pass but not actually verify anything?▼

Green tests can be broken: missing check calls, a silent sys.exit(0) in an escape branch, or assertions inside one timer-chain branch all look like passes. Confirm the printed check count matches the docstring and that the check has been seen red at least once in a counter-proof.

Should I increase the timeout when a test times out on the builder?▼

No. Raising a deadline makes every run slower and pushes the fault to the next machine. Replace wall-clock waits with the right condition, short polling intervals, or a standstill counter that moves only while the program is working.

When should I suspect the test environment instead of the code?▼

Suspect the environment first: wrong LANGUAGE locale settings, a stale VPM_SCRIPT snapshot missing sibling files, or silent SKIPPED results explain most red runs. These checks cost under a minute and should never be skipped before reading source code.