validation-artifacts

What problem does it solve?

It prevents teams from claiming “validation passed” without evidence by forcing every success claim to include reproducible, inspectable artifacts across training, inference, tests, UI, and APIs.

Core Features & Use Cases

• Mandatory proof over assertions: Collects the training logs, metrics, deterministic test evidence, visual diffs, benchmarks, and script outputs required to substantiate “passed” claims.
• Domain-specific artifact requirements: Defines tailored artifact sets for ML training (loss curves, holdout metrics), inference (predictions, confusion matrices, latency), deterministic validation (logs, reproducibility, coverage), visual validation (before/after/diff), and API/script validation (request/response samples, status codes, error cases, performance).
• Validation gating and red-flag detection: Provides clear “do not claim success” gates and common failure patterns like missing holdout evaluation, leakage suspicion, flaky tests, or unverifiable UI/API behavior.
• UI autonomous validation pattern: Supports a hack → headless screenshot → artifact commit → revert workflow to produce auditable unit-style proofs for UI modules.

Quick Start

Use the validation-artifacts skill to enforce that every “it works” claim is accompanied by the required holdout metrics, test logs, and reproducible artifacts under validation_artifacts/.