Evaluate IsaacSim checkpoints by submitting run_eval MCP tool jobs.
npx skills add https://github.com/kang-jaehyun/Mac-the-researcher --skill eval-kang-jaehyun
Please help me install this Agent Skill. Skill: eval Source: https://github.com/kang-jaehyun/Mac-the-researcher/tree/main/.claude/skills/eval Command: npx skills add https://github.com/kang-jaehyun/Mac-the-researcher --skill eval-kang-jaehyun
Enable users to evaluate a trained model's performance by running IsaacSim simulations on a given checkpoint, helping validate improvements after training.
Identify your checkpoint, ensure rlwrld_isaac is available on the target cluster, and submit the evaluation using the run_eval MCP tool.
High-intent search queries and answers about installing and using this skill.
Evaluate IsaacSim checkpoints by identifying the checkpoint via job ID, path, or HF repo ID, then submitting the evaluation through the run_eval MCP tool on a target cluster. This measures model performance by running IsaacSim simulations to validate improvements after training.
IsaacSim evaluations support local, skt L40S, and rlwrld cluster setups, preferring the skt / l40s-gpu partition for running simulations. The tool syncs if needed to ensure the target environment is ready.
Yes, rlwrld_isaac must be available on the target cluster to submit an IsaacSim checkpoint evaluation. This environment is required along with checkpoint access and the run_eval MCP tool to execute the simulation.
IsaacSim simulation evaluations require specifying the task name, data configuration, number of episodes, and the evaluation set. These parameters define how the trained checkpoint is tested during the simulation run.
Yes, you can identify the checkpoint to evaluate using a job ID, local path, or Hugging Face repository ID. The evaluation tool accesses the specified checkpoint to measure model performance across configured clusters.
IsaacSim checkpoint evaluations fail if rlwrld_isaac is not installed on the target cluster, the checkpoint is inaccessible, or the run_eval MCP tool is unavailable. Ensure all dependencies are configured on the skt L40S or rlwrld cluster before submitting.
evalstateSubmit TRL training jobs on Hugging Face Jobs and convert models to GGUF.
mqzkimAnalyze performance reports and diagnostics to propose trading system parameter adjustments.
andrewblockernstTrain and deploy neural networks in distributed E2B sandboxes.
Strategic-AutomationRun standardized LLM evaluations across 60+ benchmarks using lm-evaluation-harness.
