skills.rest
truera
trueraOFFICIAL
Explore all skills by this author →
Website𝕏@truera_aiGitHub

trulens-blocking-guardrails

Configure TruLens feedback functions as runtime guardrails to block unsafe inputs, outputs, and irrelevant context.

3.5k|335|Updated Nov 2, 2020
Search Tags:#Langchain#LlamaIndex#Guardrails#trulens#context filtering#rAG#LLM safety
One-click install
npx skills add https://github.com/truera/trulens --skill trulens-blocking-guardrails
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: trulens-blocking-guardrails
Source: https://github.com/truera/trulens/tree/main/src/core/trulens/.agents/skills/trulens-guardrails
Command: npx skills add https://github.com/truera/trulens --skill trulens-blocking-guardrails

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

LLM applications can process malicious prompts, return toxic responses, or hallucinate from irrelevant retrieved context. This Skill shows how to turn TruLens feedback functions into runtime blocking guardrails that intercept unsafe inputs and outputs and filter low-relevance documents before they reach the LLM.

Core Features & Use Cases

  • Input and Output Blocking: Use the @block_input and @block_output decorators with thresholds and fallback responses to stop criminal, harmful, or PII-containing content in plain Python apps.
  • Context Filtering for RAG: Wrap LangChain retrievers with WithFeedbackFilterDocuments or LlamaIndex query engines with WithFeedbackFilterNodes to drop chunks scoring below a context relevance threshold.
  • Testing and Monitoring: Validate guardrails with adversarial inputs and audit trigger rates and feedback scores in the TruLens dashboard.
  • Use Case: A RAG chatbot filters retrieved documents below a 0.7 relevance score so the LLM only answers from highly relevant context, reducing hallucinations.

Quick Start

Add a blocking guardrail to my TruLens app that blocks criminal inputs above a 0.9 threshold and filters irrelevant retrieved context in my LangChain RAG chain.

Frequently Asked Questions about trulens-blocking-guardrails

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I block malicious inputs in a TruLens app?▼

Use the @block_input decorator from trulens.core.guardrails.base with a criminality feedback function, a threshold, and a fallback return value. When the input score meets the threshold, the app returns the fallback response without calling the LLM.

How do I filter irrelevant documents in a LangChain RAG chain?▼

Wrap your VectorStoreRetriever with WithFeedbackFilterDocuments from trulens.apps.langchain, passing a context relevance feedback function and a threshold. Documents scoring below the threshold are dropped before the LLM synthesizes an answer.

Which feedback functions can be used as guardrails?▼

Only feedback functions that return a single float score work as guardrails. Functions returning dictionaries or chain-of-thought reasons (like relevance_with_cot_reasons) are incompatible because reasons take too long for real-time blocking.

Does TruLens guardrails support LlamaIndex?▼

Yes, use WithFeedbackFilterNodes from trulens.apps.llamaindex.guardrails to wrap a RetrieverQueryEngine. It scores each retrieved node with a feedback function and filters out nodes below the configured threshold.

How do I test that my guardrails work correctly?▼

Send adversarial inputs like harmful prompts to verify block_input returns the fallback without calling the LLM, mock toxic LLM outputs to test block_output, and ask unrelated questions to confirm context filtering removes irrelevant documents.

Related Skills

Explore More Software Engineering Skills→
trueratruera
3.5k

trulens-evaluation-setup

Configure TruLens feedback functions and selectors for LLM output evaluation.

Official
Advanced
trueratruera
3.5k

trulens-running-evaluations

Orchestrate TruLens evaluations across apps and versions with TruChain, TruGraph, and TruLlama.

Official
Intermediate
githubgithub
38.5k

phoenix-evals

Build and run evaluators for AI and LLM applications using Phoenix.

Official
Advanced
trueratruera
3.5k

trulens-instrumentation

Instrument LLM apps to capture traces for evaluation and debugging.

Official
Advanced