skills.rest
qcmuu
qcmuuCOMMUNITY
Explore all skills by this author →
GitHub

ollama

Run local LLM inference via REST API and CLI.

Updated May 2, 2026
Search Tags:#offline#rest api#inference#ollama#local llm#openai-compatible api#rag embeddings
One-click install
npx skills add https://github.com/qcmuu/AI-Research-Skills --skill ollama-qcmuu
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ollama
Source: https://github.com/qcmuu/AI-Research-Skills/tree/main/12-inference-serving/ollama
Command: npx skills add https://github.com/qcmuu/AI-Research-Skills --skill ollama-qcmuu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires ollama, httpx, openai.

What problem does it solve?

Ollama helps you avoid costly cloud inference and complicated self-hosting by letting you run open-source large language models locally with a simple API.

Core Features & Use Cases

  • Local REST and CLI inference: Generate text locally via Ollama’s built-in server endpoints and command-line workflow.
  • OpenAI-compatible chat API: Use the same client patterns and tooling you already have for OpenAI-style requests.
  • Embeddings for RAG: Produce embeddings locally for retrieval-augmented generation without sending data to external services.
  • Modelfile customization: Package models with system prompts and runtime parameters to standardize behavior across experiments.

Quick Start

Download and install Ollama, pull a model, then run it locally with the command: ollama run llama3.2.

Frequently Asked Questions about ollama

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run local LLM inference without relying on external cloud providers?▼

You can run local LLM inference without cloud providers by installing an Ollama server, pulling a model like llama3.2, and generating text via its built-in REST API endpoints. This enables offline development and privacy-sensitive workloads.

Can I use OpenAI-compatible API requests for local chat-style integrations?▼

Yes, you can use OpenAI-compatible API requests for local chat-style integrations. Ollama supports OpenAI-style requests, allowing you to reuse existing client patterns and tooling without modifying your codebase for local generation and streaming.

How do I generate RAG embeddings locally for retrieval-augmented generation?▼

To generate RAG embeddings locally for retrieval-augmented generation, use the Ollama server's embeddings endpoint. This allows you to produce embeddings locally without sending sensitive data to external services, ensuring privacy for your workloads.

Do I need an Ollama server installation to use local CLI and REST API workflows?▼

Yes, an Ollama server installation is required to use local CLI and REST API workflows. The server manages model pulling, generation, streaming, and embeddings endpoints, serving as the backend for both command-line and API interactions.

What is the best way to standardize behavior across local large language model experiments?▼

The best way to standardize behavior across local LLM experiments is using Modelfile customization. You package models with specific system prompts and runtime parameters, ensuring consistent behavior across different generation and inference runs.

Related Skills

Explore More Software Engineering Skills→
quazfentonquazfenton

cli-anything-ollama

Manage local Ollama models, generate text, and create embeddings via CLI.

Community
Intermediate
neuron-coreneuron-core
2.0k

neuron-rag-specialist

Integrate vector stores, embeddings, and document loaders into Neuron AI agents.

Official
Advanced
patrykkopycinskipatrykkopycinski
6

rag-pipeline

Set up a RAG pipeline with Elasticsearch for vector retrieval.

Community
Intermediate
zangjeicyzangjeicy

llama-cpp

Discover GGUF models on Hugging Face and run local llama.cpp inference.

Community
Advanced