hardening-pass

To validate task templates before benchmarking, run a hardening pass to detect quality issues and enforce checks for metadata, formula correctness, parameter realism, and instruction clarity.

A hardening pass is a quality gate that reviews new or updated task templates and instances to ensure validity, robustness, and reproducibility across scenarios before they are used in experiments.

You can check instruction clarity and parameter realism by providing a task directory path to a template validation process, which generates a structured report highlighting these specific quality issues.

Yes, template hardening enforces checks for proper environment and verifier integration, ensuring your task instances are correctly configured and robust before entering the benchmarking phase.

The best way to prevent invalid benchmarks is to apply a quality gate that detects and addresses issues in metadata, instructions, and environment configurations before executing experiments.

No external dependencies are required to run a task template quality review, as the hardening process operates directly on the provided template or task directory path to generate its report.