What problem does it solve? GAIA hub agents need a standardized, evidence-based eval scorecard tied to their release process, but wiring the harness-to-payload adapter, running a real eval, surfacing the scorecard, and gating releases is a multi-phase process that is easy to get wrong (e.g., hand-authored numbers, wrong README, missing CI gate). ## Core Features & Use Cases - Adapter authoring: Copy the email agent's reference gen_scorecard.py to build a harness-to-ResultPayload adapter that imports only gaia.eval.release_scorecard and records reproduction commands, dataset size, and environment metadata. - Real-eval hard gate: Run the actual benchmark (e.g., gaia eval benchmark against email fixtures) with the required env vars, serially, so the scorecard always reflects a genuine run rather than invented numbers. - Surfacing and release gating: Link SCORECARD.md from the canonical README, include it in npm files, pass it to the hub publish step, and add a scorecard-gate CI job with optional previous-release baseline comparison. - Use Case: When releasing a new version of a GAIA hub agent, use this Skill to generate its SCORECARD.md from a real eval run and wire the presence-plus-regression gate into release_agent_<id>.yml. ## Quick Start Adopt the eval scorecard for the email agent by writing the adapter, running the real benchmark, and wiring the release gate.
