skills.rest
jayll1303
jayll1303COMMUNITY
Explore all skills by this author →
GitHub

ollama-local-llm

Manage local LLMs via Ollama CLI and API with Modelfile creation.

17|3|Updated Apr 3, 2026
Search Tags:#api integration#ollama#openai-compatible#gguf#modelfile#gpu configuration#local llm
One-click install
npx skills add https://github.com/jayll1303/AIEKit --skill ollama-local-llm
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ollama-local-llm
Source: https://github.com/jayll1303/AIEKit/tree/main/.kiro/skills/ollama-local-llm
Command: npx skills add https://github.com/jayll1303/AIEKit --skill ollama-local-llm

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Running and integrating large language models locally can be complex, with challenges in model management, API serving, and hardware configuration. This Skill streamlines the process using Ollama, enabling quick deployment and interaction without relying on cloud services.

Core Features & Use Cases

  • Model Management: Pull, run, create, tag, and delete models directly from Ollama's CLI or API.
  • Custom Models: Build tailored LLMs with Modelfiles, including system prompts, parameters, and templates.
  • API Serving: Provide REST and OpenAI-compatible endpoints for application integration.
  • GPU/CPU Config: Fine-tune inference performance with environment variables for mixed CPU/GPU use.
  • Use Case: A developer can create a Python assistant model with specific prompts, then serve it locally over an OpenAI-compatible API for integration into their app.

Quick Start

Ask the ollama-local-llm skill to pull the llama3.1:8b model and run it with a greeting prompt.

Frequently Asked Questions about ollama-local-llm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a local LLM offline using Ollama and serve an OpenAI-compatible API?▼

Building custom models with a Modelfile in Ollama lets you define specific system prompts, parameters, and templates. You can then tag and run these tailored LLMs directly through the CLI or API for your development workflows.

What is the best way to import GGUF models and configure GPU settings for local inference?▼

The best way to import GGUF models and configure GPU settings is through Ollama's CLI environment variables. This allows you to fine-tune mixed CPU/GPU inference performance to satisfy low-latency, secure local requirements.

Can I use Ollama to create a Python assistant model and generate embeddings locally?▼

Yes, you can use Ollama to create a Python assistant model with specific prompts and generate embeddings locally. It supports custom Modelfile creation and serves models over a local OpenAI-compatible API for app integration.

Does running local LLMs with Ollama work without cloud services for secure inference?▼

Running local LLMs with Ollama works completely offline without cloud services, ensuring secure inference. It enables quick deployment and direct model management, satisfying low-latency requirements for private development workflows.

Related Skills

Explore More Software Engineering Skills→
IrisGoLabIrisGoLab
6

win-ai-local

Manage Ollama models and run local LLM inference on Windows 10/11.

Official
Intermediate
jayll1303jayll1303
17

text-embeddings-inference

Deploy HuggingFace TEI Docker images for embedding and re-ranking inference.

Community
Advanced
zepdbzepdb
43

zeppelin

Index and search text with Zeppelin vector store and BM25 full-text indexing.

Official
Advanced
SriRamkunamsettySriRamkunamsetty

modal-serverless-gpu

Deploy Python-configured GPU workloads on a serverless Modal platform.

Community
Advanced