Design and run structured benchmarks and acceptance criteria for AI agents.
npx skills add https://github.com/Kingxiao/agentforge --skill agentforge-benchmark
Please help me install this Agent Skill. Skill: agentforge-benchmark Source: https://github.com/Kingxiao/agentforge/tree/main/agentforge-benchmark Command: npx skills add https://github.com/Kingxiao/agentforge --skill agentforge-benchmark
This Skill helps teams test and benchmark AI agents comprehensively, including end-to-end tests, regression suites, and acceptance criteria, to reduce production risk and improve reliability.
Define your initial end-to-end task set, enable automated acceptance using LLM-as-Judge, and run SWE-bench alongside τ-bench to establish a baseline.
High-intent search queries and answers about installing and using this skill.
End-to-end testing for AI agents involves defining an initial task set and measuring statistical pass rates to verify reliability. This process reduces production risk by ensuring agents complete tasks correctly.
LLM-as-Judge is an automated acceptance testing method that calibrates agent outputs against gold standards. It provides a scalable way to evaluate agent quality without manual review for every test case.
Yes, SWE-bench and τ-bench are supported industry benchmarks for benchmarking coding and tool-use agents. Running them establishes a baseline to measure regression and guide reliability improvements.
Regression testing for RAG and Q&A assistants is fully supported, including regression case tracking and harness stability. It ensures query answering accuracy does not degrade over time.
Designing custom benchmarks for GUI browser agents involves creating a gold dataset and defining specific acceptance criteria. This validates orchestration and tool-use reliability for complex workflows.
Integration tests with Mock LLMs verify agent orchestration by isolating deterministic components and testing tool interactions. This validates that the agent workflow logic functions correctly without live model variability.
thehivegroup-aiCreate teleological plans by defining end-states, phases, tasks, and phase gates.
SimHackerCreate weighted rubrics and score outputs against defined criteria.
OpenAEC-FoundationAutomate Frappe app lifecycle from scaffolding to deployment and marketplace publishing.
HermeticOrmusEvaluates LLM outputs using automated metrics, human feedback, and LLM-as-Judge techniques.
