evaluating-cosmos-policy

You can evaluate NVIDIA Cosmos Policy by running targeted simulation benchmarks in LIBERO and RoboCasa environments to measure robotic manipulation task performance and success rates.

Yes, the evaluation supports blank-machine cluster launches, allowing you to execute robot manipulation simulations on headless GPUs in cluster environments without a display.

Robot manipulation simulation requires installing torch, mujoco, robosuite, and transformers to support the Cosmos Policy vision-language-action model evaluation workflows.

To run a LIBERO smoke evaluation, execute the provided command using uv run with the appropriate Python version and extra arguments to initiate the benchmark test.

The Skill supports both smoke and full evaluations on LIBERO benchmarks, while currently providing smoke evaluations for RoboCasa benchmarks to assess model performance.

The Skill provides reference performance benchmarking values, offering expected success rates for various robotic manipulation tasks to compare your evaluation results.