skills.rest
Jensen-Yao
Jensen-YaoCOMMUNITY
Explore all skills by this author →
GitHub

evaluating-cosmos-policy

Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa robot simulation benchmarks.

2|Updated Aug 15, 2026
Search Tags:#evaluation#simulation#robotics#vla#libero#robocasa#cosmos policy
One-click install
npx skills add https://github.com/Jensen-Yao/agents-skills --skill evaluating-cosmos-policy-jensen-yao
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluating-cosmos-policy
Source: https://github.com/Jensen-Yao/agents-skills/tree/main/skills/cosmos-policy
Command: npx skills add https://github.com/Jensen-Yao/agents-skills --skill evaluating-cosmos-policy-jensen-yao

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, mujoco, robosuite, robocasa, transformers, cosmos-policy, and includes references (resource) components.

What problem does it solve? Setting up and running NVIDIA Cosmos Policy evaluations on LIBERO and RoboCasa involves fragile GPU rendering configuration, many explicit CLI flags, and cluster-specific setup steps that are easy to get wrong. ## Core Features & Use Cases - LIBERO Evaluation Workflow: Step-by-step checklists for smoke tests and full 50-trial benchmark runs across all four LIBERO task suites using the official eval module. - RoboCasa Evaluation Workflow: Instructions for installing the Cosmos-compatible RoboCasa fork, downloading kitchen assets, and running single-task or multi-task evaluations. - Headless GPU and Cluster Setup: EGL rendering environment configuration, blank-machine cluster launch steps, and compute requirement tables for A40/A100 GPUs. - Use Case: A robotics researcher on a fresh GPU cluster node needs to reproduce the official 98.5% LIBERO average success rate; this Skill provides the exact environment variables, dependency sync commands, and eval invocations to do so. ## Quick Start Ask the agent to run a Cosmos Policy smoke evaluation on the LIBERO libero_10 task suite with one trial on a headless GPU machine.

Frequently Asked Questions about evaluating-cosmos-policy

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a Cosmos Policy evaluation on LIBERO?▼

Run the official module cosmos_policy.experiments.robot.libero.run_libero_eval via uv with the cu128 extra and libero group, passing the checkpoint path, task suite name, and trial count. Start with a one-trial smoke test before scaling to 50 trials per suite.

How to set up headless GPU rendering for MuJoCo robot simulation?▼

Set CUDA_VISIBLE_DEVICES, MUJOCO_EGL_DEVICE_ID, MUJOCO_GL=egl, and PYOPENGL_PLATFORM=egl together before launching evaluation. CUDA and EGL device indices must point to the same GPU to avoid rendering mismatches.

What GPU and VRAM does Cosmos Policy evaluation require?▼

LIBERO evaluations need roughly 16 GB VRAM and RoboCasa about 18 GB on a single A40 or A100 GPU. A one-trial LIBERO smoke run takes 5-10 minutes, while full 50-trial suites take 2-4 hours.

Why does LIBERO prompt for a config path in non-interactive shells?▼

LIBERO expects a config.yaml under ~/.libero defining benchmark_root. Pre-create this file with a small Python snippet before running batch or headless evaluations to avoid the interactive prompt.

When should I use OpenVLA or OpenPI instead of Cosmos Policy?▼

Use the fine-tuning-openvla-oft skill for OpenVLA-based policies and fine-tuning-serving-openpi for Physical Intelligence pi0 models. This skill covers only simulation evaluation of Cosmos Policy, not training or real-robot deployment.

Related Skills

Explore More Education & Research Skills→
NVIDIA-NeMoNVIDIA-NeMo
2.0k

nemotron-3-ultra-text2sql-lora

Runs Nemotron-3 Ultra Text2SQL LoRA fine-tuning as multi-node SLURM jobs via Megatron-Bridge.

Official
Advanced
agentic-inagentic-in
577

LM Evaluation Harness

Benchmark LLM performance across 60+ academic benchmarks with reproducible scoring.

Official
Advanced
rednote-machine-learningrednote-machine-learning
142

inference-autopilot

Benchmarks and tunes SGLang deployment configurations against measured workload and latency SLOs.

Official
Advanced
qcmuuqcmuu

fine-tuning-openvla-oft

Fine-tune and evaluate OpenVLA-OFT policies for robot action generation in LIBERO and ALOHA.

Community
Advanced