evaluating-cosmos-policy

You can evaluate robotics policies in LIBERO and RoboCasa by running script-driven orchestration workflows that support smoke tests, full benchmarks, and multi-task evaluations to produce baseline metrics. This requires a CUDA-enabled GPU and specific simulation dependencies.

Yes, headless GPU rendering for robotic simulation evaluations is supported using an EGL rendering environment. This allows you to execute reproducible config benchmark runs and latency profiling on remote GPU clusters without a display.

Yes, a CUDA-enabled GPU is required to run Cosmos Policy evaluation workflows. You also need an EGL rendering environment and specific software dependencies including torch>=2.1.0, mujoco>=3.0.0, robosuite>=1.4.0, and transformers>=4.40.0.

The best way to start benchmarking is to run a smoke evaluation first. This verifies your setup and provides a baseline metric before you proceed with full benchmark suites or multi-task evaluations across the simulation environments.

Latency profiling during robotic simulation benchmarking works by measuring execution times within the script-driven orchestration of evaluation workflows. It operates alongside headless EGL rendering to capture performance metrics for the policy.

Your LIBERO evaluation setup might fail during policy testing if you lack the required repositories or dependencies. You must install the cosmos-policy and robocasa repositories alongside mujoco>=3.0.0 and robosuite>=1.4.0 to ensure proper simulation execution.