vla-patterns

Name: vla-patterns
Availability: InStock
Author: uneezaismail

Convert natural language commands into ROS 2 action sequences with JSON plans.

2|Updated Nov 28, 2025

Search Tags:#vision #language #ros2 #robotics #vla

Ready to run

npx skills add https://github.com/uneezaismail/Physical-AI-Humanoid-Robotics --skill vla-patterns

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: vla-patterns
Source: https://github.com/uneezaismail/Physical-AI-Humanoid-Robotics/tree/main/.claude/skills/vla-patterns
Command: npx skills add https://github.com/uneezaismail/Physical-AI-Humanoid-Robotics --skill vla-patterns

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides Vision-Language-Action integration patterns for cognitive robotics with ROS 2, enabling end-to-end perception-to-action flows that fuse vision, language, and control.

Core Features & Use Cases

VLA Pipeline: Fuse Whisper/STT, LLM planning, and ROS 2 actions to execute robot tasks.
Vision-Language Grounding: Resolve deictic references and commands with visual context.
Use Case: A service robot translates "bring me the cup" into a pickup and delivery sequence.

Quick Start

Try a sample prompt: "Plan to navigate to (1,2) and grasp object cup_01 using VLA."

Frequently Asked Questions about vla-patterns

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I convert natural language commands into robot actions with ROS 2?▼

Vision-language-action (VLA) patterns convert natural language into ROS 2 action sequences by fusing speech recognition, LLM planning, and robot control. This Skill integrates Whisper/STT, language models, and ROS 2 to translate commands like "bring me the cup" into navigation, grasping, and delivery tasks with visual grounding.

Can I use vision-language models to resolve object references in robot commands?▼

Yes. VLA patterns resolve deictic references—like "that cup" or "the object on the left"—by grounding language in visual context. The Skill maintains an object database and uses multimodal perception to map spoken commands to specific entities, enabling accurate grasping and manipulation.

What's required to set up a cognitive robotics system with language understanding?▼

A cognitive robotics system requires ROS 2 infrastructure, a vision-language model for semantic understanding, speech-to-text capability, an object database, and a planner that enforces action ordering and feasibility. This Skill provides the integration patterns to combine these components into an end-to-end pipeline.

How does action ordering and error handling work in VLA task planning?▼

VLA planning enforces task dependencies—ensuring navigation precedes grasping—and generates JSON action plans with feasibility checks. This Skill includes replanning and limited retries when errors occur, so the robot recovers from failures without restarting the entire sequence.

Does this approach work for indoor robot navigation and manipulation tasks?▼

Yes. This Skill is designed for indoor cognitive robotics with ROS 2, supporting navigation to coordinates, object grasping and release, rotation, and speech tasks. It handles the full perception-to-action flow required for service robot applications in structured indoor environments.

What output does the VLA planner produce for robot execution?▼

The VLA planner outputs a JSON action plan specifying the sequence of ROS 2 actions—navigation, grasping, rotation, release, and speech—along with parameters, feasibility metadata, and error-handling directives. This structure enables reliable robot execution and debugging.