skills.rest
crycriM
crycriMCOMMUNITY
Explore all skills by this author →
GitHub

model-manager

Automate dynamic model swapping for the llama.cpp router during chat and completion requests.

Updated Apr 19, 2026
Search Tags:#mlops#router#proxy#model-management#llama-cpp#swap#model-loading
One-click install
npx skills add https://github.com/crycriM/hermes-skills --skill model-manager-crycrim
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-manager
Source: https://github.com/crycriM/hermes-skills/tree/main/mlops/model-manager
Command: npx skills add https://github.com/crycriM/hermes-skills --skill model-manager-crycrim

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

HTTP proxy fronting the llama.cpp router enables automatic model swapping to ensure the correct model is loaded for chat and completion requests, reducing manual intervention and downtime.

Core Features & Use Cases

  • Automatic model swapping on demand during chat/completions.
  • Background state polling to reflect external loads/unloads.
  • Health and status endpoints to observe loaded models and memory usage.

Quick Start

Run the model-manager proxy to automatically swap models for the llama.cpp router during runtime.

Frequently Asked Questions about model-manager

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automatically swap llama.cpp models during chat and completion requests?▼

You can automate llama.cpp model swapping by running an HTTP proxy in front of the router. It intercepts requests and applies memory-aware swap logic to dynamically load the correct model without manual intervention.

What is memory-aware model swapping in a llama.cpp router setup?▼

Memory-aware model swapping is a safety mechanism for the llama.cpp router that ensures single-model loading at any time. It evaluates memory constraints before swapping to prevent overloading and guarantee stable runtime operations.

How do I synchronize external model loads and unloads with my llama.cpp router?▼

You synchronize external model changes using background state polling. This process continuously monitors external loads and unloads, reflecting the actual runtime state and memory usage through dedicated health endpoints.

Does llama.cpp support automatic on-demand model swapping for multiple models?▼

Yes, when fronted by an HTTP proxy, the llama.cpp router supports automatic on-demand model swapping. This is specifically designed for scenarios where a router fronts multiple models and requires serialized swaps to guarantee safe loading.

Why do I need serialized swaps when managing multiple llama.cpp models?▼

Serialized swaps are required to guarantee safe, single-model loading at any time. Without serialized swaps, concurrent requests to the llama.cpp router could trigger overlapping load operations, causing memory exhaustion and runtime conflicts.

Related Skills

Explore More Software Engineering Skills→
PolGubauPolGubau

expo-api-routes

Create secure API endpoints in Expo Router projects on EAS Hosting.

Community
Intermediate
lawless-mlawless-m

Vram-GPU-OOM

Coordinate GPU VRAM usage across Ollama, Whisper, and ComfyUI with OOM retry and auto-unload.

Community
Advanced
jackspacejackspace
16

cloudflare-workers-ai

Run AI inference and integrate models on Cloudflare Workers AI with streaming and OpenAI-compatible endpoints.

Community
Advanced
LynxLabVNLynxLabVN

llama-cpp

Load, quantize, and run llama.cpp models for local inference.

Official
Advanced