add-llm-evals
OfficialEnhance LLM/agent app evaluation with offline and online scoring.
AuthorContextJet-ai
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill adds evaluation capabilities to LLM/agent apps, measuring output quality (correctness, faithfulness, relevance, safety) beyond just trace observation.
Core Features & Use Cases
- Offline Suite: Automates evaluation on a fixed dataset in CI.
- Online Scoring: Optionally grades production traffic for quality.
- Reference-based and Reference-free Evaluations: Supports both exact/fuzzy match, semantic similarity, and LLM-as-a-judge rubrics.
- RAG Metrics: Implements faithfulness, answer relevance, context precision, and context recall for RAG.
Quick Start
Run the offline evaluation suite on your LLM/agent app's PRs to ensure prompt and model quality.
Dependency Matrix
Required Modules
promptfoodeepevalragas
Components
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: add-llm-evals Download link: https://github.com/ContextJet-ai/awesome-llm-observability/archive/main.zip#add-llm-evals Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.