test-verify

Verifies test implementation completeness via execution gates and pins green tests as CI regression.

Updated Jun 24, 2026
One-click install
npx skills add https://github.com/Hakkadaikon/hymme --skill test-verify-hakkadaikon
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: test-verify
Source: https://github.com/Hakkadaikon/hymme/tree/main/skills/test-verify
Command: npx skills add https://github.com/Hakkadaikon/hymme --skill test-verify-hakkadaikon

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Teams often accept an implementer's claim that "all tests pass" without proof, leaving untested behaviors, flaky tests, and regressions that silently break later. This Skill closes the test-design workflow by handing the T-ID ledger to implementation, verifying completion with real execution logs instead of self-reports, and pinning green tests into CI as permanent regression coverage. ## Core Features & Use Cases - Implementation Handoff: Converts the traceability matrix (behavior → technique → test name) into failing tests for TDD-driven implementers, escalating state-machine designs to TLA+ model checking and critical properties to Lean 4 proofs when needed. - Execution Gates: Requires fresh run logs for four checks—coverage (grep every T-ID back to a real test), green suite, flakiness (repeated runs with varied seeds), and mutation testing for critical T-IDs. - Regression Pinning: Moves verified green tests into CI continuous execution, adds mutation/coverage gates for critical behaviors, and logs new defects as fresh T-IDs. - Use Case: After test-catalog assigns techniques to all T-IDs, dispatch the ledger to a coder agent, then grep each T-ID against actual test files and paste raw test output before declaring the work done. ## Quick Start Verify that every T-ID in the test ledger has a corresponding passing test by running the suite and grepping test names, then pin the green tests into CI.

Frequently Asked Questions about test-verify

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify all planned tests were actually implemented?

Reverse-lookup every T-ID in the ledger by grepping for its corresponding test name in real test files, one line at a time. A fully green suite alone does not prove coverage, since an implementer can forget T-IDs while everything still passes.

How to detect flaky tests before merging?

Run property-based, metamorphic, and concurrency tests repeatedly with varied seeds, at minimum dozens of times or via consecutive CI successes. A single green run cannot reveal flakiness caused by value nondeterminism or coordination nondeterminism.

When should I use mutation testing on my test suite?

Apply mutation testing only to critical T-IDs such as money, security, and state-transition behaviors, checking that no survived mutants indicate missing assertions. Running it on everything is considered excessive under YAGNI.

Can I trust an agent's report that all tests pass?

No. The workflow requires pasting the actual execution command and raw output, including exit codes and pass/fail summary lines, into the response. If fresh logs cannot be produced, the work is treated as incomplete.

What are the limitations of testing state machines and concurrent designs?

Tests cannot cover all interleavings of state transitions, concurrency, and ordering. The recommended path is model checking the design with TLA+, converting counterexample traces into Gherkin acceptance scenarios, then transferring those predicates into tests.