skills.rest
bonginkan
bonginkanOFFICIAL
Explore all skills by this author →
GitHub

fairy-tale-benchmark-feedback

Convert benchmark misses and successful slices into reusable agent workflow rules.

18|3|Updated Jun 13, 2026
Search Tags:#agent workflow#swe-bench#rule generation#benchmark feedback#defensive validation#miss classification#reproducible skills
One-click install
npx skills add https://github.com/bonginkan/fairy_tale --skill fairy-tale-benchmark-feedback
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: fairy-tale-benchmark-feedback
Source: https://github.com/bonginkan/fairy_tale/tree/main/plugins/fairy-tale/skills/fairy-tale-benchmark-feedback
Command: npx skills add https://github.com/bonginkan/fairy_tale --skill fairy-tale-benchmark-feedback

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill eliminates the loss of valuable, reproducible lessons from agent benchmark misses, failed work products, or successful benchmark slices that should be standardized to improve future agentic coding, closed-ended reasoning, and defensive sandbox run performance.

Core Features & Use Cases

  • Cross-Benchmark Miss Classification: Categorizes failures from SWE-Bench Pro, HLE-style closed-ended tasks, and defensive ExploitBench runs into actionable, predefined miss classes to pinpoint root causes quickly.
  • Narrow Reusable Rule Generation: Converts failure traces and observed success practices into targeted, non-contradictory rules, pruning low-value candidates before promotion to avoid regressions.
  • Use Case: After your agent fails a SWE-Bench Pro task due to an implicit contract gap, use this Skill to generate a rule to recover tacit contracts from adjacent code and fixtures before editing, preventing the same failure on similar future tasks.

Quick Start

Use the fairy-tale-benchmark-feedback skill to analyze your recent SWE-Bench Pro miss trace and generate a narrow reusable rule to prevent the same API compatibility break on future similar tasks.

Frequently Asked Questions about fairy-tale-benchmark-feedback

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert SWE-Bench Pro failure traces into reusable agent workflow rules?▼

To convert SWE-Bench Pro failure traces into reusable agent workflow rules, this Skill analyzes local work products and visible tests to classify failure signatures, attribute faults to first actionable steps, and prune contradictory candidates before promoting non-regressive rules.

What is the best way to classify benchmark misses for closed-ended reasoning tasks?▼

Classifying benchmark misses for closed-ended reasoning tasks involves categorizing failures from HLE-style tasks into predefined miss classes to pinpoint root causes, using only public task instructions and official harness artifacts to identify implicit contract gaps.

Can I use benchmark feedback to prevent regressions in agentic coding assistants?▼

Yes, benchmark feedback prevents regressions in agentic coding assistants by validating generated rules via held-out retries and promoting only non-regressive, task-agnostic rules that have been pruned of low-value candidates.

Does this approach require access to hidden answers from ExploitBench runs?▼

No, defensive ExploitBench run validation uses only public task instructions, visible tests, official harness artifacts, and local work products, strictly without accessing hidden answers or restricted data to generate targeted rules.

Why should I prune contradictory candidate rules generated from benchmark failures?▼

Pruning contradictory candidate rules generated from benchmark failures is necessary to avoid regressions, ensuring that only narrow, targeted, and non-regressive rules are promoted to improve agent performance across benchmark families.

When do I need defensive validation for reproducible benchmark slices?▼

Defensive validation for reproducible benchmark slices is needed after your agent fails a task due to an implicit contract gap, allowing you to generate a rule to recover tacit contracts from adjacent code and fixtures before editing.

Related Skills

Explore More Software Engineering Skills→
AlphaLab-USTCAlphaLab-USTC
131

bench

Run OhMyCode coding benchmarks and record pass-fail outcomes with token usage.

Community
Intermediate
Yog-SothoYog-Sotho
4

self-improving-agent-v2

Record failures, discoveries, and optimizations in a local .learnings/ journal.

Community
Advanced
seydakaratekeliseydakaratekeli

maui-current-apis

Detect deprecated .NET MAUI APIs in project source code.

Community
Advanced
mvillmowmvillmow
18

analyze-ci-failure-logs

Analyze CI failure logs to identify root causes and error patterns.

Community
Intermediate