verify-environment

Validate Castform training environments by interpreting scorecards and reward variance.

49|3|Updated Jun 27, 2025
One-click install
npx skills add https://github.com/castform-ai/benchmax --skill verify-environment
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: verify-environment
Source: https://github.com/castform-ai/benchmax/tree/main/src/benchmax/cli/scaffold/skills/verify-environment
Command: npx skills add https://github.com/castform-ai/benchmax --skill verify-environment

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you verify a Castform training environment before spending GPU credits, so you can catch reward bugs, tool issues, and weak data early.

Core Features & Use Cases

  • Baseline validation: Run and interpret castform validate as the cheap pre-launch check for a training setup.
  • Reward and variance analysis: Read scorecards, identify flat or erroring rewards, and tell whether the signal is actually discriminating.
  • Pre-launch confidence: Use it after editing an environment, data, or provider settings to decide whether the baseline is ready for launch.
  • Use case: A team updates a RAG environment and wants to confirm retrieval, reward scoring, and rollout budgets are correct before starting a GPU run.

Quick Start

Validate the current Castform environment, explain the scorecard, and tell me whether it is ready to launch.

Frequently Asked Questions about verify-environment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate a training environment before spending GPU credits?

Validate a training environment by running pre-launch checks on validation rollouts, reward behavior, and baseline health to catch reward bugs and tool issues early before GPU runs. This confirms the setup is safe to launch.

What does a flat or erroring reward signal mean on a training scorecard?

A flat or erroring reward signal on a training scorecard indicates the reward function is not discriminating properly. You must analyze reward variance and error signals to determine if the environment baseline is weak or broken.

Can I check rollout budgets and retrieval behavior for a RAG environment before launch?

You can check rollout budgets and retrieval behavior for a RAG environment before launch by running a validation baseline. This pre-GPU evaluation confirms whether reward scoring and retrieval setups are correct.

When should I run environment validation after updating my training data?

Run environment validation immediately after editing training data, provider settings, or an environment. This pre-launch check ensures the updated baseline is healthy and ready before starting expensive GPU runs.

Does baseline validation work with scaffolded agent tasks and custom training workflows?

Baseline validation works with scaffolded agent tasks, RAG setups, trace-based environments, and custom training workflows. It interprets scorecards and rollout budgets to confirm any of these setups are safe to launch.