Skills
.Work. You
Rest

local-inference

Community

Run cheap local LLM tasks with fallback

Software Engineering#mlx#ollama#litellm#local inference#model gateway#openai-compatible api#fallback routing
Authorescotilha
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill solves the problem of paying for (and depending on) cloud LLM calls for small, repetitive tasks by routing lightweight requests to local models first and automatically falling back to cloud when needed.

Core Features & Use Cases

  • Unified OpenAI-compatible gateway: Exposes local and cloud models behind a single API endpoint at http://localhost:4000/v1 so your tools don’t need provider-specific integrations.
  • Multi-backend inference with automatic fallback: Uses LiteLLM to try local tiers (MLX and/or Ollama) and then fall back to cloud backends like OpenRouter and Anthropic.
  • Practical model routing for real workflows: Supports different model “names” (e.g., local, code, claude) to map tasks like summarization/formatting vs. code generation vs. complex reasoning.

Quick Start

Ask an AI assistant to configure and start LiteLLM so your local endpoint http://localhost:4000/v1 routes the model name local to MLX and/or Ollama with cloud fallback to OpenRouter or Anthropic.

Dependency Matrix

Required Modules

None required

Components

Standard package

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: local-inference
Download link: https://github.com/escotilha/claude-public/archive/main.zip#local-inference

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.