Generate skill evaluation benchmarks from real usage history and user corrections.
npx skills add https://github.com/AMC-JTC/gbrain-1 --skill skill-autobench-amc-jtc
Please help me install this Agent Skill. Skill: skill-autobench Source: https://github.com/AMC-JTC/gbrain-1/tree/main/skills/skill-autobench Command: npx skills add https://github.com/AMC-JTC/gbrain-1 --skill skill-autobench-amc-jtc
High-intent search queries and answers about installing and using this skill.
Run skill-autobench against the target skill name. It mines invocation windows from the brain's conversation archive and session transcripts, then stages a proposed eval_contract with 4-8 replayable cases at skills/<name>/eval/autobench-<date>.md for human approval.
The skill fails closed: it emits an honest no-history report listing substrates checked and queries run, and never fabricates typical invocations. For history-less skills, use gbrain skillopt <name> --bootstrap-from-skill for a spec-derived benchmark instead.
No. It never rewrites SKILL.md, triggers, or any routing surface. The synthesized eval is staged with status PENDING-HUMAN-APPROVAL, and merging the approved eval_contract into frontmatter is an explicit human decision.
Assert over the result receipt that each named model returned a non-empty response, responses came from distinct provider endpoints, and no two differently-named models returned byte-identical output. These checks are pure assertion logic with no model calls or network cost.
Each correction is classified by cheapest durable fix: DETERMINISTIC-CODIFIABLE (convert repeated LLM fallbacks to code plus a test), PROMPT-FIXABLE (eval case plus SkillOpt run), SPEC-GAP (report to human), or ROUTING-MISS (route to routing-eval.jsonl).
No. Before staging, every mined case is rewritten onto placeholder slugs like alice-example and acme-example. Cases keep their shape and failure mode but never real entities, since staged evals live in the distributable skill repo.
Nairon-AIMine archived conversations for uncaptured patterns and corrections.
garrytanConverts raw features into tested, eval-scored, resolver-registered skill units.
weipingExplain the gbrain skillpack contract and required artifacts for validation.
garrytanGenerate skill evals from mined conversation history and user corrections.
