Validate Aeon skill outputs against an assertion manifest and detect regressions.
npx skills add https://github.com/anajuliabit/aeon --skill skill-evals-anajuliabit
Please help me install this Agent Skill. Skill: Skill Evals Source: https://github.com/anajuliabit/aeon/tree/main/skills/skill-evals Command: npx skills add https://github.com/anajuliabit/aeon --skill skill-evals-anajuliabit
Skill Evals catches quality regressions between runs by validating outputs against a per-skill assertion spec, comparing the latest results to the prior eval, and turning new failures into actionable next steps.
Run Skill Evals for a specific skill by setting var to the skill name, or leave it empty to evaluate every skill listed in evals.json.
High-intent search queries and answers about installing and using this skill.
Quality regression detection works by validating outputs against an assertion manifest and comparing the latest run against the prior eval. It classifies outcomes as new fail, fixed, or still failing to pinpoint exactly where output drift occurred.
You can validate agent outputs locally by running coverage audits via scripts against local file inputs. The system checks minimum word counts, required patterns, and numeric ranges, producing a verdict and actionable queue entirely offline.
An assertion manifest is a per-skill specification used to verify output quality. It defines required and forbidden patterns, minimum word counts, and numeric ranges to ensure autonomous operations like daily digests and monitoring logs meet expected standards.
Yes, it requires local file inputs including evals.json, aeon.yml, memory/cron-state.json, and prior eval articles. This allows unattended operations like scheduled digests and monitoring logs to be audited offline without external dependencies.
The system automatically files new issues for new failures or quality drops and can resolve previously detected issues if they are later fixed. It generates an actionable diff-driven queue and only notifies when attention is needed.
Coverage audits are limited to local file inputs and cannot make external network calls. You must have evals.json, aeon.yml, and prior eval articles available locally for the diff-driven comparison and verdict generation to function correctly.
Atrium-HermesAnalyze cron-state.json, memory logs, and run statuses to surface skill failures.
ZaxbyHubMonitor OpenCode Swarm pull requests and resolve low-risk CI failures automatically.
anajuliabitAnalyze cron state, GitHub activity, and memory items to report operational health.
Himanshu040604Automates code review requests with a dedicated subagent.
