ln-23-test-suite-auditor

Audits test suite coverage, isolation, determinism, and oracle strength as a sustainable portfolio.

556|83|Updated Oct 24, 2025
One-click install
npx skills add https://github.com/levnikolaevich/claude-code-skills --skill ln-23-test-suite-auditor
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ln-23-test-suite-auditor
Source: https://github.com/levnikolaevich/claude-code-skills/tree/main/plugins/codebase-audit-suite/skills/ln-23-test-suite-auditor
Command: npx skills add https://github.com/levnikolaevich/claude-code-skills --skill ln-23-test-suite-auditor

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Test suites often accumulate flaky, redundant, or low-value tests while leaving critical behavior unproven, making it hard to know whether a green build actually means anything. This Skill performs a read-only audit of an existing test portfolio to determine which important failures it detects, which evidence is untrustworthy or obsolete, and which additions, updates, merges, deletions, or explicit omissions produce the smallest sustainable portfolio.

Core Features & Use Cases

  • Portfolio and Coverage Audit: Maps runners, suites, and test types, traces critical behavior (money, auth, data integrity, destructive operations) to tests with real oracles, and classifies every gap as KEEP, ADD, UPDATE, MERGE, DELETE, or NO_TEST.
  • Isolation and Determinism Diagnosis: Detects shared-state leakage, order dependence, time and randomness sensitivity, and distinguishes test flakiness from genuine intermittent product defects using discriminating run matrices.
  • Oracle and Structure Review: Flags assertion-free tests, weak truthiness checks, snapshot-only proof, mock contract drift, and fragile UI locators, then issues P0-P3 findings with verified practice references.
  • Use Case: Before a major release, run the audit to learn that your payment retry logic has no failing-oracle test, three flaky suites share one global fixture, and forty snapshot tests prove nothing, then receive a prioritized remediation plan.

Quick Start

Ask the agent to audit this repository's test suite for coverage of critical behavior, flakiness, and weak assertions, and to return a verdict with prioritized portfolio actions.

Frequently Asked Questions about ln-23-test-suite-auditor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit whether my test suite actually catches real bugs?

Trace each critical behavior, such as money handling, authorization, or data integrity, to a test whose oracle would fail if that behavior broke. Line coverage and name matching are only discovery evidence; the audit requires an assertion or observable oracle per critical path.

How to diagnose flaky tests caused by order or shared state?

Run the smallest discriminating matrix: alone, in-suite, repeated with a fixed seed, shuffled, and parallel, preserving the first failing order, seed, and environment. Then check shared database, filesystem, clock, and global state for leakage between tests.

When is it safe to delete a test?

Deletion requires proof that the test's basis is obsolete or that other evidence covers every still-required behavior and failure mode with equal or better trust. Regression guards and the only proof of a rare critical edge case are never deleted for low numeric value alone.

Does high code coverage mean the test suite is trustworthy?

No. Coverage indicates execution, not proof. A line can be executed without any assertion that would fail on a defect, so the audit treats coverage as a discovery signal and inspects actual assertions and oracles before judging confidence.

What are the limitations of this test audit approach?

The audit is read-only and will not rewrite snapshots, regenerate fixtures, or implement new tests. If a required suite, environment, or oracle cannot be accessed and no static or historical fallback exists, the affected area is reported as BLOCKED rather than assumed healthy.