3d-diffuser-actor-rlbench

Automating robotic manipulation tasks with language instructions and RGB-D imagery is achieved using a 3D Vision-Language-Action policy. It combines multi-view visual data with text to predict end-effector keyposes, guiding the robot's motion planner to execute actions.

A Vision-Language-Action policy integrates with CoppeliaSim by using the PyRep backend to simulate Franka Panda robot environments. It processes RGB-D imagery and language inputs to predict keyposes, which are then executed through the simulator's motion planner.

Yes, you can use this 3D manipulation policy specifically for RLBench benchmark environments. It is designed to automate tasks within RLBench, such as opening drawers, manipulating food on a grill, and closing jars using a Franka Panda robot.

The best way to start running a vision-language-action policy for a simulated drawer opening task is by executing the provided benchmark command with the appropriate RLBench scene configuration YAML file and the specified rskill path.

Yes, you need PyTorch and OpenRAL dependencies along with ZMQ and msgpack to run 3D robotic manipulation simulations. These libraries facilitate the vision-language-action policy processing, communication, and robotic control execution.