vla-patterns

Vision-language-action (VLA) patterns convert natural language into ROS 2 action sequences by fusing speech recognition, LLM planning, and robot control. This Skill integrates Whisper/STT, language models, and ROS 2 to translate commands like "bring me the cup" into navigation, grasping, and delivery tasks with visual grounding.

Yes. VLA patterns resolve deictic references—like "that cup" or "the object on the left"—by grounding language in visual context. The Skill maintains an object database and uses multimodal perception to map spoken commands to specific entities, enabling accurate grasping and manipulation.

A cognitive robotics system requires ROS 2 infrastructure, a vision-language model for semantic understanding, speech-to-text capability, an object database, and a planner that enforces action ordering and feasibility. This Skill provides the integration patterns to combine these components into an end-to-end pipeline.

VLA planning enforces task dependencies—ensuring navigation precedes grasping—and generates JSON action plans with feasibility checks. This Skill includes replanning and limited retries when errors occur, so the robot recovers from failures without restarting the entire sequence.

Yes. This Skill is designed for indoor cognitive robotics with ROS 2, supporting navigation to coordinates, object grasping and release, rotation, and speech tasks. It handles the full perception-to-action flow required for service robot applications in structured indoor environments.

The VLA planner outputs a JSON action plan specifying the sequence of ROS 2 actions—navigation, grasping, rotation, release, and speech—along with parameters, feasibility metadata, and error-handling directives. This structure enables reliable robot execution and debugging.