Execute Loa framework evaluation suites and output JSON results.
npx skills add https://github.com/0xHoneyJar/loa-hounfour --skill eval-running
Please help me install this Agent Skill. Skill: eval-running Source: https://github.com/0xHoneyJar/loa-hounfour/tree/main/.claude/skills/eval-running Command: npx skills add https://github.com/0xHoneyJar/loa-hounfour --skill eval-running
This Skill automates the process of evaluating and benchmarking the quality of Loa skills, ensuring that framework changes do not introduce regressions and that skills maintain their expected performance.
Run the framework correctness evaluation suite by typing /eval --suite framework.
High-intent search queries and answers about installing and using this skill.
Run regression testing for skill quality evaluation by executing predefined suites against the Loa framework via a shell harness to detect performance regressions and benchmark results. You can target specific suites, tasks, or all tasks for a given skill.
Skill benchmarking is the process of evaluating skill quality against expected results. You need to update baselines when framework changes alter expected outcomes, allowing you to save new evaluation results with a documented rationale for the changes.
Execute specific evaluation tasks by targeting individual tasks or all tasks associated with a particular skill instead of running a full suite. The shell harness supports focused evaluations and outputs results in JSON format with defined exit codes for status reporting.
Yes, you can output evaluation results in JSON format for automated benchmarking. The evaluation suites provide JSON output and utilize defined exit codes for status reporting, enabling seamless integration into automated testing and continuous integration workflows.
Yes, the evaluation suite supports detecting framework regressions after updates. After deploying a new version of the Loa framework, run the regression suite to confirm existing skills function correctly and have not been negatively impacted by the changes.
0xHoneyJarRun evaluation suites against the Loa framework to detect regressions and quantify skill quality.
AGIBuildCompare BenchmarkDotNet JSON results against GitHub Actions artifact baselines.
AdeitasunaRun Loa evaluation suites to detect regressions and benchmark skill quality.
kzahediRun performance benchmarks on YARS simulations and detect regressions.
