Constrain LLM text generation with regex and context-free grammars.
npx skills add https://github.com/zhuangbiaowei/smart_bot --skill guidance-zhuangbiaowei
Please help me install this Agent Skill. Skill: guidance Source: https://github.com/zhuangbiaowei/smart_bot/tree/main/skills/guidance Command: npx skills add https://github.com/zhuangbiaowei/smart_bot --skill guidance-zhuangbiaowei
This Skill allows you to precisely control and constrain the output of Large Language Models (LLMs), ensuring generated text adheres to specific formats, grammars, and rules, thereby guaranteeing valid and structured outputs.
Use the guidance skill to generate a JSON object for a user with a name, age, and email.
High-intent search queries and answers about installing and using this skill.
You can guarantee valid JSON output from an LLM by applying constrained generation with Pythonic syntax and regular expressions. This approach forces the model to adhere to predefined formats, ensuring structured outputs and reducing post-processing needs.
Constrained text generation is a mechanism that controls LLM outputs using grammars, regular expressions, or Pythonic syntax. It enforces specific formatting rules during decoding, ensuring generated text strictly adheres to required structures like valid JSON or XML.
Yes, constrained generation works with local models via llama.cpp and Transformers backends. It also integrates with cloud APIs like OpenAI and Anthropic, allowing you to apply grammars and structured output rules across various LLM environments.
You can enforce a specific grammar or regex pattern during LLM inference by defining context-free grammars or regular expressions in your generation script. This constrains the model's token selection process to only produce text matching your defined syntax.
You can reduce LLM latency during structured text generation using a technique called token healing. This method fixes tokenization artifacts at the prompt boundary, minimizing the need for retries and accelerating multi-step generation workflows.
vasilyu1983Optimize LLM inference performance and cost with vLLM and quantization.
profmikegreeneParses pilot plans and hypotheses into structured JSON or tabular data for downstream analysis.
projectedanxEnforce syntactic and structural constraints on LLM generation using regex and grammars.
JamesFincherConstrain LLM outputs with regex and grammar for structured generation.
