What problem does it solve?
Agents and LLM outputs can be factually incorrect, inconsistent, or ungrounded; this Skill provides runtime checks and post-reasoning verification to detect hallucinations, estimate uncertainty, and prevent incorrect responses from reaching users.
Core Features & Use Cases
- Runtime Verification: Attach semantic entropy, NLI, fact decomposition, and hallucination detectors to agent builders to score and filter outputs.
- Post-Reasoning Steps: Add reflection or looped verification phases to enable self-consistency checks and automated retries until outputs meet pass thresholds.
- LLM-Scored Evaluation: Run batch evaluations across five dimensions (accuracy, relevance, completeness, safety, cost efficiency) using the @reactive-agents/eval package to measure agent quality.
- Use Case: Validate answers grounded in retrieved documents, flag risky or low-confidence responses in production pipelines, and run test suites to quantify pass rates.
Quick Start
Create an agent builder with verification enabled, set hallucination and pass thresholds, add a reflect verification step, and run an eval suite against your test cases using @reactive-agents/eval.
