Execute target skills on test inputs and score outputs against predefined criteria.
npx skills add https://github.com/tya5/reyn --skill eval-tya5
Please help me install this Agent Skill. Skill: eval Source: https://github.com/tya5/reyn/tree/main/src/reyn/stdlib/skills/eval Command: npx skills add https://github.com/tya5/reyn --skill eval-tya5
It provides a standardized way to evaluate and score AI skills based on test cases, ensuring consistent and objective assessments.
judge_phase to evaluate each phase against criteria and compiles results.Use the eval skill to automatically score and analyze a target skill's performance on a specific test case.
High-intent search queries and answers about installing and using this skill.
Automate LLM evaluation by running the target skill on test inputs and scoring its output against predefined criteria to provide objective performance metrics. This ensures consistent and standardized assessments for generated summaries or code snippets.
Deterministic scoring in AI skill assessment evaluates model outputs by executing the target skill and judging its results against predefined test cases. This process aggregates judgments to provide objective, standardized metrics for performance analysis.
Evaluate AI skill performance automatically by defining specific test inputs and predefined criteria for the target skill. The system executes the skill on these inputs, captures outputs, and uses a judge phase to compile objective scoring results.
Yes, you can use automated evaluation to score language model summaries against predefined standards. The system executes the target skill on test inputs, captures generated outputs, and applies judgment aggregation to provide objective performance metrics.
Yes, automated skill evaluation works without external dependencies. It executes the target skill internally, captures outputs, and applies judgment aggregation against predefined criteria to compile objective scoring results for performance analysis.
Using predefined criteria for LLM judgment aggregation limits evaluation to known test cases and expected standards. It ensures objective scoring for defined scenarios but may not fully capture nuanced performance variations outside the specified assessment boundaries.
selcukyucelGenerate structured evaluation datasets with binary rubric scoring for AI outputs.
enigmaicon-engScore or compare LLM outputs with rubric-based LLM-as-judge evaluation and structured JSON results.
JaykaijackAutomate drafting, testing, and iterative improvement of AI skills and SKILL.md descriptions.
boazcstrikeImplement LLM-as-a-Judge evaluation systems with pairwise comparison and bias mitigation.