challenge-gen

Generate targeted training challenges from agent struggle profiles with pipe-format metadata.

Updated Feb 20, 2026

Search Tags:#diagnostics #edge-cases #training #challenge #active-learning #agent-improvement #commit-replay

Ready to run

npx skills add https://github.com/supertyrelle/pelley --skill challenge-gen-supertyrelle

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: challenge-gen
Source: https://github.com/supertyrelle/pelley/tree/main/skills/challenge-gen
Command: npx skills add https://github.com/supertyrelle/pelley --skill challenge-gen-supertyrelle

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill turns an agent's diagnosed weak spots into targeted practice challenges, making improvement concrete and repeatable instead of vague.

Core Features & Use Cases

Profile-aware challenge design: Uses /diagnose-agent output or basic profiling to identify weaknesses, gaps, strengths, and calibration needs.
Grounded scenario generation: Pulls from real codebase edge cases and commit history to create realistic edge-case and replay tasks.
Challenge packaging: Writes a structured challenge set with scenarios, acceptance criteria, hidden traps, and provenance for downstream learning workflows.

Quick Start

Use the challenge-gen skill with an agent name to generate a calibrated set of training challenges after running /diagnose-agent.

Frequently Asked Questions about challenge-gen

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I generate training challenges from agent diagnostics?▼

To generate training challenges from agent diagnostics, you use the challenge-gen Skill with an agent name after running a profile diagnosis. It processes the agent struggle profile to produce targeted edge-case practice and commit-replay tasks for code-focused agents.

What is commit-replay training for code-focused agents?▼

Commit-replay training for code-focused agents is a practice method that uses real codebase commit history to create realistic edge-case scenarios. It turns diagnosed weak spots into targeted, repeatable practice tasks instead of vague improvement goals.

How do I create grounded edge-case scenarios from a codebase?▼

You create grounded edge-case scenarios by feeding an agent struggle profile or basic profiling input into a challenge generator. The system pulls from real codebase edge cases and commit history to package scenarios with acceptance criteria and hidden traps.

Do I need to run a diagnosis before generating practice challenges?▼

Yes, running a diagnosis before generating practice challenges is required to identify weaknesses, gaps, and calibration needs. The challenge generation workflow validates agent ownership and uses the diagnostic output to select grounded scenarios for targeted improvement.

Can I use basic profiling input instead of full diagnostics for challenge generation?▼

Yes, you can use basic profiling input instead of full diagnostics for challenge generation. The system accepts both detailed diagnostic output and basic profiling data to identify agent weaknesses and design calibrated edge-case training scenarios.

What is the best way to structure active-learning scenarios for agent improvement?▼

The best way to structure active-learning scenarios for agent improvement is to package them as a structured challenge set. This includes specific scenarios, acceptance criteria, hidden traps, and provenance metadata to support downstream learning workflows.