langsmith-evaluator
OfficialBuild scalable, code-driven LangSmith evaluators.
Software Engineering#evaluation#structured-output#langsmith#evaluator#offline-evaluation#online-evaluation#code-evaluator
Authorlangchain-ai
Version1.0.0
Installs0
System Documentation
What problem does it solve?
LangSmith-based evaluation pipelines for agent outputs with reproducible, code-driven evaluators that ensure consistent scoring across tasks and datasets.
Core Features & Use Cases
- Define deterministic run evaluators in Python or TypeScript to validate outputs, trajectories, and schema conformity.
- Upload offline evaluators attached to datasets or online evaluators attached to projects for real-time quality checks.
- Align evaluation with LangSmith workflows, enabling structured scoring and debugging insights.
Quick Start
Capture agent outputs, write a small code evaluator in Python or TypeScript, and upload it to LangSmith to validate results against your dataset or project.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: langsmith-evaluator Download link: https://github.com/langchain-ai/skills-benchmarks/archive/main.zip#langsmith-evaluator Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.