forge-subagent-eval

What problem does it solve?

Prevent silent multi-agent failures by ensuring a parent/orchestrator does not treat a subagent’s claims as ground truth without verification, especially for irreversible actions.

Core Features & Use Cases

• Shape and budget validation: Reject malformed or out-of-scope returns (e.g., non-JSON or schema mismatches) before reading content.
• Hallucination defenses: Cross-check high-impact claims like filenames, URLs, quotes, and numerical statements against real artifacts.
• Refusal handling and bounded retry: Treat refusals as signals, refine the brief once, cap retries, and decide fallback/escalation after failures.
• Partial-result recovery: Use successful fan-out outputs while separately surfacing failures, then deduplicate/aggregate before judging.
• Decision audit trail: Log the parent’s accept/retry/reject decisions with reasons per subagent call for debugging and accountability.

Quick Start

Use the forge-subagent-eval skill when your orchestrator receives a subagent response and you must verify whether to accept, retry with a sharper brief, or fall back instead of acting irreversibly.