trulens-evaluation-setup

To configure TruLens evaluation feedback functions, define your providers, selectors, and aggregation strategies. This setup enables consistent measurement of LLM outputs across experiments and allows you to compare versions across runs.

TruLens selectors allow you to target specific inputs, contexts, and traces from your application for score aggregation. They work across RAG and agent-based workflows to isolate and evaluate specific data points within your LLM app.

You do not need OpenAI specifically, but you optionally require provider integrations like OpenAI to run feedback functions. The core requirement is the TruLens core library to handle the evaluation setup and measurement.

Setting up instrumentation for TruLens evaluation metrics requires proper application tracing to generate RECORD_ROOT spans. Once instrumented, you can run a pilot evaluation to view initial scores and track improvements across runs.

The best way to measure and compare RAG application outputs is by configuring tailored feedback functions and aggregation strategies. This approach streamlines measurement across experiments and enables consistent comparison across different runs.