Route LLM requests to local MLX or Ollama models with cloud fallback.
npx skills add https://github.com/escotilha/claude-public --skill local-inference
Please help me install this Agent Skill. Skill: local-inference Source: https://github.com/escotilha/claude-public/tree/main/skills/local-inference Command: npx skills add https://github.com/escotilha/claude-public --skill local-inference
This Skill solves the problem of paying for (and depending on) cloud LLM calls for small, repetitive tasks by routing lightweight requests to local models first and automatically falling back to cloud when needed.
Ask an AI assistant to configure and start LiteLLM so your local endpoint http://localhost:4000/v1 routes the model name local to MLX and/or Ollama with cloud fallback to OpenRouter or Anthropic.
High-intent search queries and answers about installing and using this skill.
To run local LLM inference with automatic cloud fallback, route your requests through a unified gateway that tries local MLX or Ollama models first, then falls back to cloud providers like OpenRouter or Anthropic when local fails.
Yes, you can use an OpenAI-compatible API for local model routing by mapping different model names to specific tasks. This exposes local and cloud backends behind a single endpoint so your tools avoid provider-specific integrations.
To configure the LiteLLM proxy for MLX and Ollama, set up a local endpoint at localhost:4000/v1 that maps model names like 'local' or 'code' to your local tiers, establishing a fallback chain to cloud backends.
The best way to reduce cloud LLM costs for small tasks is routing lightweight requests to local models first. By using a gateway with automatic fallback, you only pay for cloud calls when local tiers cannot handle the request.
LiteLLM supports fallback routing to OpenRouter and Anthropic by configuring a fallback chain. When local MLX or Ollama tiers fail or are unavailable, the gateway automatically routes the request to these cloud backends.
You should not use a local inference gateway for complex reasoning tasks that exceed local model capabilities. While it handles chat, summarization, and bounded code generation, heavy workloads may immediately fall back to cloud providers.
wenermeConfigure LiteLLM proxy server, virtual keys, and spend tracking.
vm0-aiAccess DeepSeek chat, reasoning, and code generation via curl-based API requests.
SpillwaveSolutionsConfigure LLM providers for agent-memory summarization via interactive wizard.
iulianditaArchitect and review production AI applications with LLM integrations and RAG pipelines.
