honest-ship-eval

Model readiness validation requires executing multi-suite adversarial evaluation gates across smoke, held-out, and OOD suites. This ensures production claims are grounded in verifiable parse, fidelity, and component recall metrics rather than soft metrics.

Adversarial testing for deployment gates involves running rigorous evaluation against out-of-distribution and held-out suites to prevent premature deployment. It enforces strict parsing and fidelity thresholds to ensure claims are backed by verifiable evidence.

Constrained decoding evaluation requires strict adherence to decoding invariants and version-stamped evidence contracts during the assessment process. This validates structural and semantic density requirements defined in the project's goal law.

Automated ship gates enforce non-negotiable readiness validation by applying multi-suite structural and adversarial testing. They prevent models relying on hidden data leakage from passing by requiring strict adherence to fidelity thresholds.

Soft metrics lack the verifiable evidence needed for production readiness, allowing premature deployment of models with hidden data leakage. Rigorous evaluation gates replace them with objective parse, fidelity, and component recall metrics.

Honest-ship-eval operates as a standalone script component without external dependencies to validate ship readiness. It executes the full evaluation suite against your current model run to generate version-stamped evidence contracts.