outlines

Generate structured JSON, regex, and Pydantic-constrained outputs from local LLMs.

Updated May 4, 2026
One-click install
npx skills add https://github.com/Junkfooooood/hermes-governance --skill outlines-junkfooooood
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: outlines
Source: https://github.com/Junkfooooood/hermes-governance/tree/main/skills/mlops/inference/outlines
Command: npx skills add https://github.com/Junkfooooood/hermes-governance --skill outlines-junkfooooood

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires outlines, transformers, vllm, pydantic, and includes references (resource) components.

What problem does it solve? LLMs often produce malformed JSON or free-form text that breaks downstream parsing. This Skill constrains token generation at the logit level using finite state machines, so outputs always match your JSON schema, Pydantic model, or regex pattern without retry loops. ## Core Features & Use Cases - Schema-Guaranteed Generation: Convert Pydantic models or JSON schemas into grammars that filter invalid tokens during decoding, guaranteeing valid structured output. - Multiple Local Backends: Run constrained generation on Transformers, llama.cpp, or vLLM models, with multi-GPU and quantization support for production throughput. - Typed Generators: Use built-in generators for choices, integers, floats, regex patterns, and full JSON objects with nested models, enums, and field constraints. - Use Case: Extract structured company information (name, founded year, industry, employees) from hundreds of unstructured text snippets into validated Pydantic objects for a data pipeline. ## Quick Start Ask the AI to use Outlines with a Pydantic model to extract structured fields from your text and return a validated JSON object.

Frequently Asked Questions about outlines

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate JSON from an LLM that matches a Pydantic model?

Define a Pydantic BaseModel with your fields, load a model with outlines.models.transformers, then create a generator with outlines.generate.json(model, YourModel). The output is guaranteed to validate against your schema because invalid tokens are filtered during generation.

What is the difference between Outlines and Instructor for structured output?

Outlines constrains token sampling with finite state machines for zero-overhead generation and works best with local models. Instructor uses API-based models with automatic retrying on validation failure, making it better suited for OpenAI-style hosted endpoints.

Does Outlines support vLLM for high-throughput inference?

Yes, Outlines supports vLLM via outlines.models.vllm, including tensor parallelism across multiple GPUs and AWQ or GPTQ quantization. This makes it suitable for production batch processing with high concurrent sequence counts.

Can I use Outlines with OpenAI API models?

Outlines has limited OpenAI support through outlines.models.openai, but some advanced features like FSM-based token filtering do not work with API models. Full structured generation capabilities require local backends like Transformers, llama.cpp, or vLLM.

How do I constrain LLM output to match a regex pattern?

Use outlines.generate.regex with your pattern, such as a phone number format like [0-9]{3}-[0-9]{3}-[0-9]{4}. The generator compiles the regex into a state machine and only samples tokens that keep the output valid.

Why is my structured generation slow with deeply nested Pydantic models?

Deeply nested schemas create larger finite state machines, increasing compilation and per-step filtering cost. Flatten your model structure where possible, reuse cached generators, and use vLLM for batch workloads to improve throughput.