Dispatch targeted challenges to a named agent and evaluate output.
npx skills add https://github.com/supertyrelle/pelley --skill challenge-run-supertyrelle
Please help me install this Agent Skill. Skill: challenge-run Source: https://github.com/supertyrelle/pelley/tree/main/skills/challenge-run Command: npx skills add https://github.com/supertyrelle/pelley --skill challenge-run-supertyrelle
This Skill runs targeted challenge sets against a named agent and turns the results into objective evidence about performance, trap awareness, and confidence calibration.
Run /challenge-run with the target agent name after /challenge-gen to evaluate that agent against the latest challenge set.
High-intent search queries and answers about installing and using this skill.
You can evaluate agent performance by dispatching self-contained challenge prompts, scoring outputs against concealed ground truth, and writing persistent reports to track calibration and growth over time.
Hidden-trap evaluation scores agent solutions against concealed traps and ground truth without leaking evaluation criteria to the agent, measuring task completion alongside confidence calibration.
Track confidence calibration by comparing self-reported confidence against actual outcomes, saving results as persistent reports under memory/agents/<name>/challenges/ for downstream learning.
Yes, you must generate a challenge set first. Run /challenge-gen to create the targeted scenarios, then execute /challenge-run with the agent name to evaluate performance against that latest set.
No, agent challenge execution requires serial processing. The evaluation mandates sequential challenge dispatch to accurately measure task completion, hidden-trap detection, and confidence calibration without interference.
The evaluation outputs pipe-format context and persistent files saved under memory/agents/<name>/challenges/, enabling downstream active-learning loops to read the benchmark results and track improvement.
supertyrelleGenerate targeted training challenges from agent struggle profiles with pipe-format metadata.
FilipcsupkaAutomate persistent memory management for AI agents using AgentDB.
luuspoo-createDispatch parallel agents to investigate and fix independent software failures.
supertyrelleDiagnose agent weaknesses, generate targeted challenges, and update learnings from evaluated rounds.
