prompt-guard

Detect prompt injection and jailbreak attempts by loading the Prompt-Guard-86M model with transformers and torch to score text, producing BENIGN, INJECTION, or JAILBREAK probabilities for user messages before LLM inference.

Filter retrieved RAG documents for malicious content in batches by applying the sequence classification model to score multiple passages simultaneously, flagging and discarding suspicious retrieved text based on configurable thresholds.

Configure thresholds for blocking unsafe prompts by setting a score cutoff for the JAILBREAK or INJECTION probabilities output by the classifier, allowing you to gate user messages and block requests exceeding your chosen safety limit.

Run a jailbreak detection model for LLM security by installing the transformers and torch dependencies, which are required to load the 86M parameter sequence classification model and process text inputs for probability scoring.

Prompt injection detection works for vetting third-party API and web content by applying lower thresholds to the classifier's probability scores, allowing you to flag and discard suspicious external passages before they enter your application context.

The difference between injection and jailbreak in LLM security scoring is defined by the model's output labels: INJECTION covers attempts to override instructions, while JAILBREAK targets attempts to bypass safety guardrails entirely.