realitykit-ar-companion

Build interactive AR companions with RealityKit on iOS and visionOS.

Updated Dec 2, 2025
One-click install
npx skills add https://github.com/domocarroll/wayfinding-companion --skill realitykit-ar-companion
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: realitykit-ar-companion
Source: https://github.com/domocarroll/wayfinding-companion/tree/main/RealityKit_Enhanced
Command: npx skills add https://github.com/domocarroll/wayfinding-companion --skill realitykit-ar-companion

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

RealityKit AR Companion Development enables developers to rapidly build interactive, persistent AR companions within iOS and visionOS apps, reducing integration friction and time-to-first-interaction.

Core Features & Use Cases

  • ECS-based architecture: Entities, components, and systems to model companion behavior, animation, and interactions.
  • Tracking and animation: Body and hand tracking, facial morph targets, procedural motions, and skeletal rigs for lifelike character motion.
  • AI-driven behavior: State machines, behavior trees, and utility AI patterns for responsive, believable companions.
  • Lifecycle & environment: Persistent state management, scene lifecycle integration, spatial audio, and AR anchoring to enable real-world persistence and immersion.
  • Use Case: Create an AR companion that follows the user, reacts to gestures, talks, and interacts with their space while preserving state across sessions.

Quick Start

Begin by initializing RealityKit on iOS/visionOS, attach a character to a detected horizontal plane, and experiment with basic tracking, animation, and AI patterns using the provided examples and references.

Frequently Asked Questions about realitykit-ar-companion

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build an interactive AR companion in RealityKit that follows users and reacts to gestures?

To build an interactive AR companion in RealityKit, implement ECS architecture for companion behavior, use body and hand tracking for gesture reactions, and apply state machines or behavior trees to drive responsive, lifelike character motions and interactions.

What is the best way to manage state persistence for AR companions across iOS sessions?

The best way to manage AR companion state persistence across iOS sessions is integrating scene lifecycle management and AR anchoring, which enables virtual characters to preserve their state and maintain real-world persistence between interactions.

Can I use RealityKit ECS architecture for AI-driven character behavior on visionOS?

Yes, you can use RealityKit ECS architecture to model AI-driven character behavior on visionOS by implementing utility AI patterns, behavior trees, and state machines that process entity components to generate responsive and believable companion reactions.

How does body and hand tracking work for animating AR characters in RealityKit?

Body and hand tracking in RealityKit captures physical user motions and maps them onto skeletal rigs and facial morph targets, enabling procedural motions and lifelike character animation for interactive AR companions.

Do I need ARKit integrations to implement spatial audio and entity lifecycle management for AR companions?

Yes, you need Apple RealityKit and ARKit integrations to implement spatial audio, AR anchoring, and entity lifecycle management, ensuring robust environment interaction and real-world persistence for virtual characters.

How do I anchor a virtual character to a detected horizontal plane in an iOS AR app?

To anchor a virtual character to a detected horizontal plane in an iOS AR app, initialize RealityKit, detect the plane using ARKit, and attach the companion entity to the plane anchor to ensure it interacts accurately with the physical space.