problem-quality-pipeline

Problem quality validation is automated by coupling a Problem Curator agent with pytest-problem-reviewer to detect issues and apply automatic fixes. This pipeline orchestrates an iterative validation loop that re-runs the reviewer after curator updates to confirm improvements until a clean PASS is achieved.

An iterative improvement pipeline for QA problems detects quality issues like ambiguous contracts or incomplete tests and applies curator-driven fixes. It enforces explicit input-output contracts, maintains markdown integrity, and re-validates across multiple iterations until achieving a passing state.

Batch processing multiple problems is supported with configurable max iterations. The pipeline handles multiple QA problems described in the problem space, applying scenario refinement, difficulty templating, and guardrail validation across each problem independently.

Report generation is handled automatically by the pipeline, producing a final report, iteration history, and pipeline log. These outputs capture the curator-driven fixes, reviewer validation results, and the full progression of the iterative improvement loop.

The pipeline operates without external dependencies, relying on its integrated Problem Curator and pytest-problem-reviewer components. It runs via CLI commands to initialize or progress fixes on a given problem, maintaining markdown validation throughout the auto-heal workflow.

Guardrail validation is needed when QA problem statements require enforced explicit contracts for inputs and outputs. The pipeline applies this validation during scenario refinement and difficulty templating to ensure problem quality meets structural and content standards before final approval.