Author and certify eval tasks measuring API detail accuracy.
npx skills add https://github.com/tangle-network/agent-runtime --skill generate-eval
Please help me install this Agent Skill. Skill: generate-eval Source: https://github.com/tangle-network/agent-runtime/tree/main/skills/generate-eval Command: npx skills add https://github.com/tangle-network/agent-runtime --skill generate-eval
This Skill assists in authoring and certifying execution-certifiable, search-discriminating eval tasks for a given target, ensuring accurate API detail measurement.
Use the generate-eval skill to create an eval task for the library '[email protected]' and output the candidate JSON object to 'task.json'.
High-intent search queries and answers about installing and using this skill.
To create eval tasks for API detail measurement, you author tasks with specific schema fields, setup commands, and reference code targeting a library version or release notes, then output the candidate JSON object.
Certifying an eval task against grounding and discrimination gates means an independent runtime validates that the candidate task is accurately grounded and effectively discriminates whether web search aids a coding agent against the real target.
Yes, you can generate eval tasks based on release notes and repository documentation, as the task authoring process explicitly focuses on using library versions, repositories, or release notes to ensure accurate API detail measurement.
Authoring a search-discriminating eval task requires providing specific schema fields, setup commands, and reference code to ensure the task is execution-certifiable and can accurately measure if web search aids a coding agent.
You validate that an eval task is execution-certifiable by using the runtime to certify the candidate tasks against both grounding and discrimination gates, ensuring it meets the required standards before testing against the real target.
rivaldiekaptrrrDefine and run eval suites to validate AI task performance.
exploreomniExecute Omni CLI eval prompts and score generated query JSON against expected results.
jayhjenkinsRestate code review feedback and verify against the repository before implementing changes.
JhabbigDesign evaluation frameworks for LLM applications with automated and human scoring.
