fix-solution

Automated iterative debugging applies a test-and-fix loop that runs initial evaluations, analyzes failures, and makes minimal changes to benchmark solutions until tests pass. It re-runs evaluations after each repair to confirm the outcome.

Cross-checkpoint propagation applies fixes made at one checkpoint to future checkpoints to maintain consistency across the benchmark. This ensures repairs remain valid as the solution progresses through iterative testing.

No external dependencies are required to run the iterative test-and-fix workflow. The process operates independently to evaluate failures, apply minimal changes, and re-run evaluations without needing additional libraries.

During failure analysis, guardrails keep fixes minimal and prompt for user guidance if tests seem incorrect. This prevents the workflow from overwriting or bypassing test logic that may contain intentional validation rules.

Invoke the workflow with a snapshot path, problem name, and checkpoint index to begin the automated evaluation cycle. The system will run initial checks, analyze failures, and iteratively apply minimal fixes.

If solutions keep failing, the failure analysis mechanism may identify that tests themselves are incorrect and prompt for user guidance. The disciplined workflow ensures changes remain minimal to avoid introducing new errors.