skills.rest
cbusillo
cbusilloCOMMUNITY
Explore all skills by this author →
GitHub

local-llm

Inventory, chat with, and benchmark local LLM endpoints through LM Studio APIs.

5|1|Updated May 5, 2026
Search Tags:#local llm#model inventory#lm studio#openai-compatible api#model benchmarking#private inference#local agent
One-click install
npx skills add https://github.com/cbusillo/codex-skills --skill local-llm-cbusillo
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: local-llm
Source: https://github.com/cbusillo/codex-skills/tree/main/skills/local-llm
Command: npx skills add https://github.com/cbusillo/codex-skills --skill local-llm-cbusillo

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires PyYAML, and includes scripts (resource) and references (resource) components.

What problem does it solve? Running private or sensitive prompts against cloud APIs risks data exposure, while managing local models manually is error-prone. This Skill provides a controlled way to inventory, call, benchmark, and curate locally hosted or trusted-network OpenAI-compatible models without leaking private data. ## Core Features & Use Cases - Endpoint and Model Inventory: List API-visible models and LM Studio native runtime state, with locality and trust classification for each endpoint. - Bounded Local Chat: Send one-shot prompts with JIT or explicit model loading, warm-up probes, TTL hints, context-length and flash-attention verification, and instance-scoped cleanup. - Benchmarking and Curation: Benchmark role or model candidates with short probes and maintain a public-safe curated model index with roles, strengths, and tested notes. - Isolated Local Agent Runs: Execute a Codex or Codex Lab agent against a verified local model in a sandboxed, credential-free child session. - Use Case: Before sending a private rollout report to a model, run an inventory check, warm up the curated rollout_scout role via JIT loading, and send the prompt knowing the endpoint is localhost and the output stays local. ## Quick Start Ask the assistant to list the models available on the local LM Studio endpoint and then send a short test prompt through the rollout_scout role.

Frequently Asked Questions about local-llm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I list models available in LM Studio from the command line?▼

Run the lm_studio_inventory.py script with uv to query the /v1/models endpoint and, for LM Studio providers, the native runtime endpoint. It reports endpoint locality, trust, model IDs, and loaded instance counts in text or JSON.

How do I send a prompt to a local LLM with LM Studio?▼

Use lm_studio_chat.py with a --role or --model flag, optionally adding --load-policy jit_chat, --ttl, and --warmup. The warm-up probe absorbs cold-load latency and records the served model before your private prompt is sent.

Can I use trusted LAN or remote private endpoints with local LLM scripts?▼

Yes, endpoints can be configured as localhost, trusted_lan, or remote_private in the private .local/local-llm.yaml config. Non-localhost endpoint URLs are redacted in output, and cloud endpoints are refused for isolated agent execution.

Why does my local model return no assistant content?▼

Large reasoning models can consume the entire max_tokens budget on reasoning, leaving no visible reply. Increase max_tokens before declaring the model unusable; the scripts surface this as an empty-content error.

What is the difference between jit_chat and api_explicit load policies?▼

jit_chat relies on LM Studio's per-request TTL hint for just-in-time loading, while api_explicit calls the native /api/v1/models/load endpoint with verified context length and flash attention, binds chat to the returned instance ID, and can unload it afterward.

When should I not use local LLM endpoints?▼

Avoid local endpoints when the runtime is untrusted, the endpoint is misconfigured, or the task requires capabilities the installed models lack. The scripts never silently fall back to cloud; they fail visibly and keep private inputs local.

Related Skills

Explore More Software Engineering Skills→
eduardtomasekeduardtomasek

grepai-embeddings-lmstudio

Configure LM Studio as the local embedding provider for GrepAI semantic code search.

Community
Basic
BEIRUXBEIRUX

summarize

Summarize or extract text from URLs, podcasts, and local files.

Official
Intermediate
sumitakesumitake

agent-readiness

Evaluates whether an agent-runtime combination is ready for a proposed role.

Community
Intermediate
MetriLLMMetriLLM
5

metrillm

Benchmark local LLMs for performance, quality, and RAM compatibility.

Official
Intermediate