foundation-models-on-device

On-device text generation in iOS uses Apple FoundationModels to run private language models locally. This enables offline inferencing without sending user data to the cloud, ensuring privacy while generating text responses.

Structured output from an LLM is achieved using the @Generable feature in FoundationModels. This provides typed data directly from the on-device model, enabling reliable downstream processing and safe integration with your app's logic.

Custom tool calling is supported by FoundationModels on-device. You can define domain-specific actions that the local LLM can trigger during real-time user interactions, extending the model's capabilities beyond standard text generation.

Streaming LLM responses progressively to the UI is handled using snapshot streaming. This FoundationModels feature provides real-time UI updates by delivering generated content snapshots as the on-device model produces them.

FoundationModels requires model availability checks before use. Enforcing these checks alongside per-session management ensures a robust UX by verifying the on-device model is ready before initializing generation tasks.