ollama

Run local LLM inference via REST API and CLI.

Updated May 2, 2026

Search Tags:#offline #rest api #inference #ollama #local llm #openai-compatible api #rag embeddings

Ready to run

npx skills add https://github.com/qcmuu/AI-Research-Skills --skill ollama-qcmuu

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: ollama
Source: https://github.com/qcmuu/AI-Research-Skills/tree/main/12-inference-serving/ollama
Command: npx skills add https://github.com/qcmuu/AI-Research-Skills --skill ollama-qcmuu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires ollama, httpx, openai.

What problem does it solve?

Ollama helps you avoid costly cloud inference and complicated self-hosting by letting you run open-source large language models locally with a simple API.

Core Features & Use Cases

Local REST and CLI inference: Generate text locally via Ollama’s built-in server endpoints and command-line workflow.
OpenAI-compatible chat API: Use the same client patterns and tooling you already have for OpenAI-style requests.
Embeddings for RAG: Produce embeddings locally for retrieval-augmented generation without sending data to external services.
Modelfile customization: Package models with system prompts and runtime parameters to standardize behavior across experiments.

Quick Start

Download and install Ollama, pull a model, then run it locally with the command: ollama run llama3.2.

Frequently Asked Questions about ollama

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run local LLM inference without relying on external cloud providers?▼

You can run local LLM inference without cloud providers by installing an Ollama server, pulling a model like llama3.2, and generating text via its built-in REST API endpoints. This enables offline development and privacy-sensitive workloads.

Can I use OpenAI-compatible API requests for local chat-style integrations?▼

Yes, you can use OpenAI-compatible API requests for local chat-style integrations. Ollama supports OpenAI-style requests, allowing you to reuse existing client patterns and tooling without modifying your codebase for local generation and streaming.

How do I generate RAG embeddings locally for retrieval-augmented generation?▼

To generate RAG embeddings locally for retrieval-augmented generation, use the Ollama server's embeddings endpoint. This allows you to produce embeddings locally without sending sensitive data to external services, ensuring privacy for your workloads.

Do I need an Ollama server installation to use local CLI and REST API workflows?▼

Yes, an Ollama server installation is required to use local CLI and REST API workflows. The server manages model pulling, generation, streaming, and embeddings endpoints, serving as the backend for both command-line and API interactions.

What is the best way to standardize behavior across local large language model experiments?▼

The best way to standardize behavior across local LLM experiments is using Modelfile customization. You package models with specific system prompts and runtime parameters, ensuring consistent behavior across different generation and inference runs.