foundation-models-on-device

On-device text generation with FoundationModels is enabled by creating a session within the framework to generate text locally. This approach ensures privacy-first processing and offline capabilities without sending data to external servers.

Yes, structured output is supported using the @Generable macro for on-device LLM generation. This allows you to define specific data structures and receive formatted responses directly from the FoundationModels framework.

FoundationModels supports both streaming and tool calling for iOS development. These features allow real-time token generation and interaction with external functions, enabling complex generable structures and dynamic app workflows.

On-device LLM limitations include token window size constraints and the need to satisfy device availability checks. You must verify the device supports FoundationModels and manage context length to ensure successful local generation.

You must perform on-device availability checks to verify FoundationModels support before starting a session. This ensures the iOS device has the necessary hardware and software capabilities for local LLM generation.

Using FoundationModels provides a privacy-first method for offline generative AI in iOS applications. Data never leaves the device during generation, ensuring user privacy while supporting structured outputs and streaming.