privatelm-cross-platform-llm-client

What problem does it solve?

This skill helps you build privacy-focused LLM chat apps by enabling fast on-device (offline-first) inference while keeping a cloud fallback for higher capability or when local models aren’t available.

Core Features & Use Cases

• Unified local + cloud chat: Route requests to on-device GGUF inference (via llama.cpp) or to cloud providers like OpenAI, Anthropic Claude, Google Gemini, and Kimi.
• Multimodal support (text + vision): Run image-capable local models (e.g., Qwen2-VL) or send images to cloud vision-capable endpoints.
• Device-aware performance tuning: Detect RAM/GPU tier and auto-select inference parameters (context size, threads, GPU layers) to fit low-end Android devices.
• Persistent sessions & offline-first storage: Save conversations and settings locally using Hive, including model downloads and chat history management.
• Use case example: Build a Flutter app where users chat with a local GGUF model on Android when offline, but automatically switch to a cloud model to handle complex queries when connectivity exists.

Quick Start

Instruct the AI to generate a Flutter integration plan for PrivateLM in your app, including local GGUF setup, provider switching logic, and how to persist chat sessions with Hive.