hermes-atropos-environments

To build an RL environment for multi-turn tool-calling agent training, you extend HermesAgentBaseEnv and implement required methods like setup, get_next_item, format_prompt, compute_reward, and evaluate so Atropos can train against your environment.

The evaluate() method runs the full HermesAgentLoop, executing tools, tool parsing, and ToolContext verification instead of single-turn chat completions. You extract final responses from AgentResult.messages to score multi-turn rollouts accurately.

You can use LLM-judge scoring or deterministic sandbox verification via ToolContext terminal execution for agent evaluation. The environment extracts final responses from AgentResult.messages and wires reward logging with wandb metrics correctly.

Atropos supports offline trajectory generation through its process CLI mode, which outputs JSONL files. It also offers serve mode for the training loop and evaluate mode for tool-using benchmark runs, keeping results separate from training metrics.

You log reward metrics to wandb by implementing the wandb_log method required by HermesAgentBaseEnv. This ensures Atropos correctly tracks and records training metrics during tool-enabled rollouts.

Single-turn chat completions skip the HermesAgentLoop, failing to execute tools or verify ToolContext. Multi-turn tool-enabled rollouts are required to properly evaluate agent behavior and compute accurate rewards for Atropos training.