skills.rest
Aradotso
AradotsoOFFICIAL
Explore all skills by this author →
Website𝕏@aradotsoGitHub

privatelm-cross-platform-llm-client

Build a cross-platform Flutter LLM client with local GGUF inference and cloud fallback.

4|Updated May 16, 2026
Search Tags:#llm#multimodal#flutter#gguf#on-device inference#cloud api fallback#hive persistence
One-click install
npx skills add https://github.com/Aradotso/devtools-skills --skill privatelm-cross-platform-llm-client
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: privatelm-cross-platform-llm-client
Source: https://github.com/Aradotso/devtools-skills/tree/main/skills/privatelm-cross-platform-llm-client
Command: npx skills add https://github.com/Aradotso/devtools-skills --skill privatelm-cross-platform-llm-client

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps you build privacy-focused LLM chat apps by enabling fast on-device (offline-first) inference while keeping a cloud fallback for higher capability or when local models aren’t available.

Core Features & Use Cases

  • Unified local + cloud chat: Route requests to on-device GGUF inference (via llama.cpp) or to cloud providers like OpenAI, Anthropic Claude, Google Gemini, and Kimi.
  • Multimodal support (text + vision): Run image-capable local models (e.g., Qwen2-VL) or send images to cloud vision-capable endpoints.
  • Device-aware performance tuning: Detect RAM/GPU tier and auto-select inference parameters (context size, threads, GPU layers) to fit low-end Android devices.
  • Persistent sessions & offline-first storage: Save conversations and settings locally using Hive, including model downloads and chat history management.
  • Use case example: Build a Flutter app where users chat with a local GGUF model on Android when offline, but automatically switch to a cloud model to handle complex queries when connectivity exists.

Quick Start

Instruct the AI to generate a Flutter integration plan for PrivateLM in your app, including local GGUF setup, provider switching logic, and how to persist chat sessions with Hive.

Frequently Asked Questions about privatelm-cross-platform-llm-client

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local LLM inference in a Flutter app with cloud API fallback?▼

To run local LLM inference in a Flutter app with cloud API fallback, integrate llama.cpp for on-device GGUF processing and route to OpenAI or Claude when offline capabilities are exceeded. This enables offline-first chat with cloud provider switching.

What is offline-first LLM chat and how does multimodal vision support work on-device?▼

Offline-first LLM chat prioritizes local GGUF inference via llama.cpp to function without connectivity. Multimodal vision support works by running image-capable local models like Qwen2-VL or sending images to cloud vision endpoints when online.

Can I use Flutter to build a cross-platform LLM client for iOS and Android?▼

Yes, you can use Flutter to build a cross-platform LLM client for iOS and Android. The implementation integrates llama.cpp-based native inference for both platforms, enabling unified local and cloud chat sessions with multimodal support.

How do I persist LLM chat sessions locally in a Flutter application?▼

You persist LLM chat sessions locally in a Flutter application using Hive. This offline-first storage approach saves conversations, manages model downloads, and retains chat history and user settings directly on the device.

How do I tune llama.cpp inference parameters for low-end Android devices?▼

You tune llama.cpp inference parameters for low-end Android devices using device-aware performance tuning. The client detects RAM and GPU tier to auto-select context size, threads, and GPU layers that fit the available hardware.

What is the best way to switch between local GGUF models and cloud providers in a chat app?▼

The best way to switch between local GGUF models and cloud providers in a chat app is implementing provider-switching logic. This dynamically routes requests to on-device inference or cloud APIs like Gemini and Kimi based on connectivity.

Related Skills

Explore More Software Engineering Skills→
xiaoliwanshuixiaoliwanshui
21

flutter-interoperating-with-native-apis

Integrate Flutter apps with native platform APIs via FFI and Platform Channels.

Community
Advanced
twjarviszyra-webtwjarviszyra-web
3

llama-cpp

Run local LLM inference from GGUF models on CPUs and GPUs.

Community
Intermediate
tanaka-mambingetanaka-mambinge

flutter-caching-data

Implement offline-first caching with SQLite or Hive in Flutter apps.

Community
Intermediate
NeuroSkill-comNeuroSkill-com
11

neuroskill-llm

Run a local llama.cpp server for OpenAI-compatible chat and GGUF model management.

Official
Advanced