skills.rest
Yar177
Yar177COMMUNITY
Explore all skills by this author →
WebsiteGitHub

apple-ai

Implement on-device AI features using Foundation Models, Vision, Speech, Natural Language, Core ML, and MLX Swift.

Updated Jun 17, 2026
Search Tags:#rag#speech recognition#swift#apple intelligence#foundation models#on-device ai#core ml
One-click install
npx skills add https://github.com/Yar177/appleNativeSkills --skill apple-ai-yar177
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: apple-ai
Source: https://github.com/Yar177/appleNativeSkills/tree/main/apple-ai
Command: npx skills add https://github.com/Yar177/appleNativeSkills --skill apple-ai-yar177

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Building AI features on Apple platforms requires choosing between many frameworks (Foundation Models, Vision, Speech, Natural Language, Core ML, MLX) and handling availability checks, model asset downloads, context-window limits, and privacy constraints correctly. This Skill provides the routing guidance, API patterns, and failure-mode handling needed to ship on-device AI features on iOS 26 / macOS 26 without sending user data to a server. ## Core Features & Use Cases - Foundation Models LLM integration: LanguageModelSession usage, @Generable/@Guide guided generation, streaming snapshots, tool calling, dynamic GenerationSchema, transcripts, and context-window management. - Vision, Speech, and Natural Language: OCR and detection with Swift Vision requests, SpeechAnalyzer/SpeechTranscriber speech-to-text with asset installation, NLTagger sentiment/NER, and NLEmbedding/NLContextualEmbedding for semantic similarity. - On-device RAG and custom models: chunk-embed-retrieve-ground pipelines, Core ML conversion and quantization with coremltools, and MLX Swift / llama.cpp inference for open-source LLMs on Apple Silicon. - Use Case: Build a private notes assistant that transcribes voice memos with SpeechTranscriber, embeds them with NLContextualEmbedding, retrieves relevant chunks via cosine similarity, and answers questions with a grounded Foundation Models prompt, all on device. ## Quick Start Ask the agent to add an on-device summarization feature to your SwiftUI app using Foundation Models with availability checks and a fallback for unsupported devices.

Frequently Asked Questions about apple-ai

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I get structured output from Apple Foundation Models in Swift?▼

Annotate a Swift type with @Generable and constrain fields with @Guide, then call session.respond(to:generating:) with your type. The model returns a strongly-typed value instead of free text, and you can stream partial results with streamResponse.

How do I check if Apple Intelligence is available on a device?▼

Switch over SystemLanguageModel.default.availability before using Foundation Models. Handle cases like appleIntelligenceNotEnabled, modelNotReady, and deviceNotEligible with distinct fallbacks such as prompting the user to enable Apple Intelligence or hiding the feature.

What is the difference between MLX Swift and llama.cpp for on-device LLMs?▼

MLX Swift uses Apple Silicon unified memory for higher sustained throughput and loads mlx-community models, while llama.cpp runs GGUF models with broader cross-platform support. Choose MLX for performance on Apple devices and llama.cpp when you need a model only published as GGUF.

Does SpeechTranscriber work offline on iOS?▼

Yes, SpeechAnalyzer and SpeechTranscriber run on device, but transcription models are downloadable assets. Use AssetInventory.assetInstallationRequest to download and install them before transcribing, and check SpeechTranscriber.isAvailable and supportedLocales first.

How do I build on-device RAG with Apple frameworks?▼

Chunk your content into small passages, embed each with NLContextualEmbedding, store vectors locally, and retrieve top-k chunks by cosine similarity at query time. Then inject the retrieved text into a Foundation Models prompt instructing it to answer only from the provided context.

Why does my Core ML model run slowly on the Neural Engine?▼

Broad RangeDim flexible shapes often fall back to CPU or GPU; use EnumeratedShapes so the Neural Engine can optimize each shape. Also ensure you emit mlprogram format, call model.eval() before tracing, and profile per-layer placement with MLComputePlan.

Related Skills

Explore More Software Engineering Skills→
yashpalsince2004yashpalsince2004

photokit

Implement photo picking, camera capture, and media handling in iOS apps.

Community
Advanced
travisjneumantravisjneuman
88

ios-development

Develop native Apple platform apps using Swift, SwiftUI, and UIKit.

Community
Intermediate
ios-agentios-agent
10

sirikit

Integrate Siri voice interactions and Shortcuts into iOS/watchOS apps using SiriKit.

Official
Advanced
VYRE-StudiosVYRE-Studios

llama-cpp

Run GGUF model inference on CPU and non-NVIDIA hardware.

Official
Advanced