skills.rest
reason-machines
reason-machinesOFFICIAL
Explore all skills by this author →
Website𝕏@reasonmachinesGitHub

privatelm-cross-platform-llm-client

Implements Flutter chat apps with local GGUF inference and cloud LLM API fallback.

4|Updated May 16, 2026
Search Tags:#llm#Flutter#GGUF#llama.cpp#cloud fallback#on-device inference#chat app
One-click install
npx skills add https://github.com/reason-machines/devtools-skills --skill privatelm-cross-platform-llm-client-reason-machines
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: privatelm-cross-platform-llm-client
Source: https://github.com/reason-machines/devtools-skills/tree/main/skills/privatelm-cross-platform-llm-client
Command: npx skills add https://github.com/reason-machines/devtools-skills --skill privatelm-cross-platform-llm-client-reason-machines

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Building privacy-focused AI chat apps on mobile requires juggling on-device model inference, multiple cloud provider APIs, and device-specific performance tuning, which is complex to wire together from scratch. ## Core Features & Use Cases - Local On-Device Inference: Runs GGUF models via llama.cpp on Android and iOS with GPU acceleration and automatic device-tier detection for optimal context size, threads, and GPU layers. - Cloud API Fallback: Streams responses from OpenAI, Anthropic Claude, Google Gemini, and Kimi with a unified interface for switching providers at runtime. - Multimodal & Offline-First: Supports vision inputs (Qwen2-VL locally, GPT-4V/Gemini in cloud) and persists chats, models, and settings locally with Hive. - Use Case: A developer building a privacy-first Android chat app can auto-detect device RAM, download a recommended quantized model, and fall back to Claude when local inference fails. ## Quick Start Ask the AI to scaffold a Flutter chat app using PrivateLM that loads a local GGUF model on Android with automatic fallback to the OpenAI API when local inference is unavailable.

Frequently Asked Questions about privatelm-cross-platform-llm-client

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run GGUF models locally in a Flutter app?▼

Use the InferenceService to load GGUF models via llama.cpp on Android or iOS, specifying context size, thread count, and GPU layers. Android requires minSdk 28 and arm64-v8a ABI; iOS uses Metal for GPU acceleration.

How to switch between local LLM and cloud API providers?▼

Call switchProvider on the ChatController with 'local', 'openai', 'claude', or 'gemini'. The controller routes generation to either the local inference engine or the corresponding cloud streaming endpoint, with API keys stored in Hive.

Does PrivateLM support iOS and web platforms?▼

iOS supports local inference through Metal GPU acceleration with a minimum deployment target of iOS 12. Web is cloud-only because local llama.cpp inference is not available in browsers.

How do I optimize local inference for low-end Android devices?▼

Use DeviceInfoService to detect RAM and classify the device into ultra, high, mid, or low tiers. Low-tier devices get a 1024 context size, 2 threads, and 8 GPU layers, paired with small models like Qwen2-1.5B or TinyLlama-1.1B.

Why does local model loading fail on my device?▼

Loading fails when the model file exceeds roughly 70 percent of available device RAM or the file path is invalid. Validate file size against detected RAM before loading and run a short test generation to confirm the model works.

Can I use vision models with both local and cloud inference?▼

Yes, VisionService supports Qwen2-VL locally by passing base64-encoded images in chat messages, and GPT-4V or Gemini in cloud mode using their respective image payload formats.

Related Skills

Explore More Software Engineering Skills→
eliottbusinesseliottbusiness

llama-cpp

Run llama.cpp GGUF model inference locally with Hugging Face Hub discovery.

Community
Advanced
rodydavisrodydavis
58

using-fastlane-in-flutter-and-ci

Automate Flutter builds and deployments to App Store and Google Play with Fastlane.

Community
Advanced
tuliopc23tuliopc23

coreml-diag

Diagnose CoreML model loading, performance, and conversion issues on Apple devices.

Community
Advanced
altrupetsaltrupets

machine-learning

Integrate TensorFlow Lite and Firebase ML Kit into Flutter applications.

Official
Advanced