What problem does it solve?
After a CTF competition, it is hard to tell whether failures came from individual hard challenges or from systemic weaknesses in the agent's architecture. This Skill reads the agent's challenge working directories, computes solve-rate statistics, and converts raw logs into prioritized, system-level improvement recommendations.
Core Features & Use Cases
- Solve-rate statistics: Scans all challenge directories under agent-work/ctf/, detects solved flags in progress.md and findings.log, and produces per-category solve-rate tables.
- Failure diagnosis: For each unsolved challenge, extracts stop reason, final phase, dead ends, tool usage patterns, and knowledge-base search behavior.
- Root-cause classification: Maps failures to ten systemic root-cause codes (e.g., KB_MISS, JS_BLIND, LOOP_SAME_VECTOR) and outputs P0-P3 prioritized optimization suggestions tied to concrete system components.
- Use Case: After a competition, run the analysis to discover that three unsolved challenges had version numbers but never triggered a knowledge-base search, then get a P0 recommendation to fix the kb_search trigger conditions in the orchestrator prompt.
Quick Start
Analyze this competition's results and tell me which system components need improvement.
