nemo-guardrails

Install nemoguardrails, define safety flows in a RailsConfig using Colang syntax, then wrap your LLM with LLMRails. Calls to rails.generate() automatically apply input and output rails before and after model invocation.

Define user message patterns matching injection attempts like "ignore previous instructions" in a Colang flow, then map them to a refusal response. The rail blocks matching prompts before they reach the LLM.

NeMo Guardrails is a programmable framework supporting multiple safety mechanisms including jailbreak detection, PII filtering, and fact-checking. LlamaGuard is a standalone moderation model that can be integrated as one rail within NeMo Guardrails.

No, NeMo Guardrails runs on CPU, though a GPU improves latency. An NVIDIA T4 with 4-8GB VRAM is recommended when integrating LlamaGuard; pattern-matching rails add under 1ms overhead.

False positives occur when detection thresholds are too strict. Raise the jailbreak score threshold (for example from 0.5 to 0.8) in your Colang flow, or refine the user intent patterns to reduce overly broad matches.

Run checks in parallel using Colang's parallel block so toxicity, jailbreak, and PII checks execute concurrently. Typical total overhead is 100-500ms, with LLM-based checks taking 50-200ms each when serialized.