Run a skill on a test input, self-grade, and rewrite with A/B decisions.
npx skills add https://github.com/toobulkeh/claude-whetstone --skill whetstone
Please help me install this Agent Skill. Skill: whetstone Source: https://github.com/toobulkeh/claude-whetstone/tree/main Command: npx skills add https://github.com/toobulkeh/claude-whetstone --skill whetstone
This Skill helps you stop iterating on one-off prompts and instead turn a workflow into a durable, self-improving skill artifact that gets better with each run.
Run the whetstone loop on the path to your target skill file and provide a real test input, then answer the A/B questions it asks before it folds changes.
High-intent search queries and answers about installing and using this skill.
To improve coding agent prompts, you run a target skill on a real test input, self-grade the output against its own rules, and then rewrite the skill file with user-approved A/B decisions. This iterative loop refines one-off prompts into durable, versioned instruction artifacts.
Skill distillation is the process of generating a structured skill.md draft from prior context when the target skill file does not exist. It distills historical workflow context into a baseline instruction file, which is then immediately improved through the self-grading and A/B refinement loop.
Self-grading works by executing a cold-run on a real test input and evaluating the generated output against the target skill's own established rules. It provides grade evidence to identify unspecified decisions, blocking further refinement until A/B ambiguities are resolved.
Yes, you can use this to resolve A/B ambiguities by surfacing constrained choices whenever the target skill leaves a decision unspecified. It blocks the rewriting process to ask you A/B questions, ensuring the next version matches your specific preference.
Limitations include enforcing a strict turn budget for the agent's execution and requiring a real test input to run the cold-run. Additionally, the Fold phase edits only the target skill file, meaning external system prompts or unrelated files cannot be modified during the refinement process.
No, you do not need an existing skill file to start refining agent workflows. If the target skill is missing, the system distills prior context into a structured skill.md draft and then immediately begins improving it through the self-grading and A/B loop.
zhnnky329Run robustness, sensitivity, error, and baseline checks on modeling outputs.
cpfcoachingManage and optimize agent context with structured hierarchy and rule files.
flutterOrchestrate automated evaluation of agent skills using isolated subagents and rubrics.
Undertone0809Convert conversation workflows into reusable agent skills with SKILL.md files.
