Score candidate artifacts against user-defined criteria and generate an ASI.
npx skills add https://github.com/2389-research/simmer --skill simmer-judge
Please help me install this Agent Skill. Skill: simmer-judge Source: https://github.com/2389-research/simmer/tree/main/skills/simmer-judge Command: npx skills add https://github.com/2389-research/simmer --skill simmer-judge
This skill provides a structured judge that scores a candidate artifact against user-defined criteria on a 1-10 scale and outputs an actionable ASI to steer the next generator round.
Provide a candidate artifact and 2-3 criteria; the judge will score the artifact on a 1-10 scale and emit an ASI for the next generator round.
High-intent search queries and answers about installing and using this skill.
Scoring a candidate artifact against custom criteria requires providing the artifact and 2-3 defined rubrics. The judge applies a 1-10 scale and emits an actionable ASI to steer the next generator round.
An actionable ASI (Actionable Steering Instruction) provides concrete feedback after artifact evaluation. It uses criteria scores and iteration context to guide the next generator round in a hybrid evaluation workflow.
Yes, you can incorporate external test metrics into artifact scoring using runnable evaluator mode. This hybrid evaluation assimilates evaluator outputs alongside judge results to guide iteration across artifacts.
Evaluating a seed artifact and generating iteration feedback uses judge-only mode. It scores the seed against defined rubrics on a 1-10 scale and outputs a concrete ASI for the subsequent generator round.
Yes, the judge requires iteration context including iteration number and seed calibration. This context is necessary to properly reason about artifact scores and generate the ASI for the next generator round.
petrSimonidesXartEvaluate technical approaches against project constraints with documented evidence.
2389-researchGenerate improved artifact versions from ASI feedback in the simmer loop.
Dianshu-LiaoImplement LLM evaluation with automated metrics and LLM-as-judge patterns.
fagemxDiagnose and score a Game Design Document against gstack-game Review rubrics.
