benchmark-coder

Generate executable benchmark scripts and helper modules from approved plans.

2|Updated Jan 31, 2026
One-click install
npx skills add https://github.com/MRiabov/Problemologist-AI --skill benchmark-coder
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: benchmark-coder
Source: https://github.com/MRiabov/Problemologist-AI/tree/main/.agents/skills/benchmark-coder
Command: npx skills add https://github.com/MRiabov/Problemologist-AI --skill benchmark-coder

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Translates approved benchmark handoffs into executable benchmark_script.py and helper modules, preserving planner contracts and enabling robust validation, simulation, and evidence collection.

Core Features & Use Cases

  • Translates the approved planner handoff into benchmark_script.py and helper modules that implement the contracted benchmark.
  • Validates and simulates revisions before handoff, ensuring compliance with geometry and inventory constraints.
  • Generates review-ready evidence and documentation (e.g., plan_refusal, renders, and scene data) for reviewers.

Quick Start

Run the benchmark coder on the approved handoff to generate benchmark_script.py and supporting artifacts, then validate and simulate the revision before handoff.

Frequently Asked Questions about benchmark-coder

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert an approved benchmark plan into an executable Python script?

To convert an approved benchmark plan into an executable script, the benchmark coder translates the planner handoff into benchmark_script.py and helper modules. This process preserves inventory contracts and ensures the resulting code is import-safe for execution.

What is the process for validating and simulating benchmark script revisions before handoff?

Validating and simulating benchmark script revisions involves using provided helpers to test code against geometry and inventory constraints. This ensures compliance and fixes simulation failures before the final handoff to reviewers.

How do I generate review-ready evidence and documentation for a benchmark handoff?

Generating review-ready evidence for a benchmark handoff requires producing supporting artifacts like renders, scene data, and plan_refusal documentation. These outputs provide reviewers with the necessary validation and simulation proof.

Can I use the benchmark coder to fix validation failures in an existing benchmark script?

Yes, you can use the benchmark coder to fix validation or simulation failures in an existing benchmark script. It applies provided helpers to validate revisions and ensures the updated code complies with geometry and inventory constraints.

Do I need an approved plan before generating benchmark scripts and helper modules?

Yes, an approved plan is required before generating benchmark scripts. The benchmark coder operates specifically after plan approval to translate the handoff into executable benchmark_script.py and supporting modules.