foundation-models-on-device

On-device language models integrate into iOS apps using the FoundationModels framework to provide local text generation and offline capability. This approach runs entirely on-device, ensuring user data privacy while supporting text generation and summarization tasks.

Structured output from an on-device LLM is achieved using the @Generable attribute. This feature structures generated text into defined models, allowing your iOS app to directly consume typed data instead of parsing raw text strings.

Real-time LLM responses stream in iOS using snapshot streaming. This mechanism provides incremental updates during text generation, enabling your UI to render tokens progressively as the on-device model constructs the final output.

The FoundationModels framework supports custom tool calling for domain-specific actions. This allows your on-device iOS app to execute predefined functions, interact with local data, and handle complex offline workflows triggered by the language model.

On-device language models operate under specific token limits and require compatible iOS hardware. They are designed for local execution and offline scenarios, meaning generation capacity is bounded by the device's available memory and framework constraints.