evaluations
CommunityCoordinate end-to-end evaluations with confidence.
Authorvpittamp
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Workflow-Builders need a unified platform to create, run, and grade evaluations across datasets, evals, and runs, with provenance for SWE-bench benchmarks and code-eval templates.
Core Features & Use Cases
- End-to-end lifecycle for datasets, evals, runs, and run items with bidding grader configurations and provenance tracking.
- Supports code-eval templates (HumanEval+/MBPP+/BigCodeBench) and SWE-bench style benchmarks, with per-agent grading and async/sync graders.
- Public UI/API surfaces for managing datasets, evals, runs, and benchmark provenance across templates and workflows.
Quick Start
Create a new evaluation run from the UI or API to seed a dataset and begin grading.
Dependency Matrix
Required Modules
None requiredComponents
references
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: evaluations Download link: https://github.com/vpittamp/nixos-config/archive/main.zip#evaluations Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.