apple-on-device-ai

Name: apple-on-device-ai
Availability: InStock
Author: Felipe Schmitz

Configure on-device AI on Apple Silicon with Foundation Models, Core ML, and LLM runtimes.

Updated Apr 19, 2026

Search Tags:#foundation-models #on-device #tool-calling #apple-silicon #llama.cpp #core-ml #mlx-swift

Ready to run

npx skills add https://github.com/femitz/flyby --skill apple-on-device-ai-femitz

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: apple-on-device-ai
Source: https://github.com/femitz/flyby/tree/main/.agents/skills/apple-on-device-ai
Command: npx skills add https://github.com/femitz/flyby --skill apple-on-device-ai-femitz

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill enables private, offline AI on Apple devices by guiding the selection, deployment, and optimization of on-device ML models using Foundation Models, Core ML, MLX Swift, and llama.cpp.

Core Features & Use Cases

On-device language modeling and inference with Foundation Models (LanguageModelSession, @Generable, SystemLanguageModel)
Core ML model conversion, optimization, and deployment (coremltools, quantization, palettization, pruning) for Neural Engine acceleration
Running open-source LLMs on Apple Silicon via MLX Swift and llama.cpp (GGUF) for high-throughput inference
Multi-backend orchestration and tool calling to enable private, on-device AI workflows

Quick Start

Use Foundation Models to create a LanguageModelSession on an Apple Silicon device and generate a short response locally.

Frequently Asked Questions about apple-on-device-ai

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run on-device AI locally on Apple Silicon without an internet connection?▼

On-device AI on Apple Silicon runs offline by orchestrating Foundation Models, Core ML, MLX Swift, and llama.cpp for private, local inference. This combination enables tool calling and local execution without requiring network connectivity.

What is the best way to convert and optimize open-source LLMs for the Apple Neural Engine?▼

The best way to optimize LLMs for the Neural Engine is using coremltools for model conversion, applying quantization, palettization, and pruning. This workflow ensures the model is highly optimized for Core ML deployment and hardware acceleration on Apple devices.

How do I use Foundation Models for tool calling in a local Swift app?▼

You can use Foundation Models for tool calling in a local Swift app by initializing a LanguageModelSession and using @Generable with SystemLanguageModel. This enables structured, private, on-device AI workflows and multi-backend orchestration.

Does this approach support running GGUF models with llama.cpp on Apple Silicon?▼

Yes, running GGUF models with llama.cpp is fully supported on Apple Silicon for high-throughput inference. The skill guides runtime orchestration across llama.cpp, MLX Swift, and Foundation Models to coordinate cross-backend operations.

When should I choose MLX Swift over Core ML for on-device inference?▼

Choose MLX Swift for running open-source LLMs with high-throughput inference on Apple Silicon, and use Core ML when you need Neural Engine acceleration via coremltools optimization. The skill helps coordinate these backends based on your specific workflow needs.

Do I need an active network connection to use SystemLanguageModel for inference?▼

No, you do not need an active network connection to use SystemLanguageModel. By combining Foundation Models with local runtimes like MLX Swift and llama.cpp, the system delivers completely private, offline AI operation.