add-benchmark

What problem does it solve?

This Skill provides a complete, end-to-end workflow for adding a new benchmark to NeMo-Gym, including scaffolding a resources server, preparing data, configuring resources, and validating end-to-end rollouts.

Core Features & Use Cases

• Scaffold a resources server for a new benchmark with a consistent project structure.
• Define data formats (Gym JSONL), test patterns, and YAML config patterns to ensure reproducible experiments.
• Validate benchmarks with an integrated test and baseline workflow, including end-to-end smoke tests and reward profiling.

Quick Start

Run the NeMo-Gym scaffold and wire datasets, configs, and tests to add a new benchmark.