aps-judge-submission

Treat each verdict as a symptom, not a root cause: WA can stem from misreading, wrong logic, or edge cases; TLE from complexity, slow implementation, or language choice. Store candidate causes with confidence levels rather than asserting a single diagnosis.

Hosting a judge means owning sandboxing infrastructure like isolate, cgroups, and seccomp for safe arbitrary code execution, which is costly. Integrating external judges shifts the burden to sync reliability, rate limits, API policies, and handling delayed or stale verdict states.

An attempt is the user's full solving effort including reading, thinking, hints, and review, while a submission is only the observable code sent to a judge. Products should link submissions to attempts rather than modeling progress from submissions alone.

In-contest submissions run only against pretests, and full system testing happens after the contest, possibly reversing earlier accepts. Hacks from other participants can also break pretest-passing solutions, so stored verdicts need a provisional or can-still-change flag.

Some judges give Python or Java multipliers while others like Codeforces apply one uniform limit, so a Python TLE may reflect language overhead rather than wrong complexity. Products need both the language and the platform's policy to interpret TLE correctly.

It defines product requirements, domain distinctions, and risks but never produces algorithm solutions, competitive-programming code, or debugging of a specific user submission. Diagnosis of concrete wrong answers is handed off to adjacent AI assistance or explanation experts.