forge-subagent-eval

You can prevent orchestrators from acting on unverifiable claims by applying shape and budget validation, internal consistency checks, and spot-verification of external claims like URLs, filenames, and quotes before accepting the subagent output.

Handle subagent refusals by treating them as signals, refining the task brief once, capping retries, and then deciding whether to fall back or escalate after the bounded retry fails.

Prevent irreversible actions by cross-checking high-impact claims like filenames and numerical statements against real artifacts, then logging the parent orchestrator's accept, retry, or reject decision per call.

Orchestrators fail silently on malformed JSON when they lack shape and budget validation to reject non-JSON or schema mismatches before reading the content and acting on unverifiable claims.

Yes, you can use partial-result recovery in fan-out tasks by aggregating successful outputs while separately surfacing failures, then deduplicating the results before judging the final output.

You need audit logging for multi-agent decision making to log the parent orchestrator's accept, retry, or reject decisions with reasons per subagent call for debugging and accountability.