checks-that-fail-open

Restate the pass condition as a specific presence rather than an absence. Derive an enumeration from the artifact under test and assert a property of each item, so an empty result indicates a broken instrument rather than a clean run.

A grep that errors or matches nothing prints zero lines, and zero lines contain no bad string, so the check passes vacuously. Reconcile the sweep against an expected inventory first, and treat an empty result as a repair job on the instrument.

Read the exit code. Pytest uses 0 for success, 2 for collection errors, and 5 for no tests collected, while grepping output for 'passed' or 'failed' misfires on test names and summary wording. Also reconcile the collected count against what you expected to run.

If both sides of the comparison filter on the property the defect would fail, the failing case sits outside both sides and the test cannot fail. Enumerate every item and assert the property of each one instead of comparing numbers about the thing.

Rarely; the Skill argues absence checks should be restated as presences. If you must keep one, run it against a known-bad input to prove it fires, quote the actual output in the record, and state explicitly what was checked and what could not be reached.