Design end-to-end eval specs for LLM applications using Langfuse.
npx skills add https://github.com/tavva/ben-claude-plugins --skill eval-design
Please help me install this Agent Skill. Skill: eval-design Source: https://github.com/tavva/ben-claude-plugins/tree/main/plugins/eval-designer/skills/eval-design Command: npx skills add https://github.com/tavva/ben-claude-plugins --skill eval-design
The Eval Design skill helps teams architect robust, production-ready evaluations for LLMs, translating user goals into concrete specs that a coding agent can implement with Langfuse.
Describe a production-grade eval design for the user's goal and provide a Langfuse-ready spec.
High-intent search queries and answers about installing and using this skill.
Design production-grade LLM evals by creating end-to-end evaluation specs using Langfuse. This process defines success criteria, generates binary PASS/FAIL rubrics, and produces output conforming to the Eval Design Document Template for direct SDK implementation.
Create judge prompts and golden datasets by mapping specific failure modes to evaluation types like LLM-as-judge or human annotation. This generates concrete dataset requirements and rubric specifications aligned with Langfuse SDK prompts.
The best way to structure rubrics for LLM-as-judge evaluations is to define binary PASS/FAIL criteria mapped to specific failure modes. This approach translates user goals into a concrete Eval Design Document Template ready for production.
Yes, you can use this eval-design approach for code-based evaluations and human annotation. It maps distinct evaluation types to specific failure modes, ensuring the generated specs support diverse assessment methods alongside LLM-as-judge setups.
Yes, you need Langfuse to implement the generated evaluation specs. The output is specifically designed to conform to the Eval Design Document Template and be directly implementable with the Langfuse SDK for production contexts.
Productfculty-aipmDesign evaluation suites with three-layer run plans for AI product features.
maragudkBuild and validate LLM-as-Judge evaluators for automated quality assessment.
roanbrasilTrace LLM calls and track token costs across LangSmith, LangFuse, and Phoenix.
ethan-rioManage human annotation workflows for Arize LLM observability projects.
