agentic-eval

To evaluate and improve AI-generated code through self-critique, supply your task and predefined quality criteria to trigger an iterative refinement loop. The system generates an initial output, critiques it against your standards, and applies targeted fixes until thresholds are met or iteration limits are reached.

The evaluator-optimizer pattern for refining LLM outputs separates the generation and evaluation phases to ensure clearer responsibilities and more reliable results. It enforces deterministic evaluation steps and applies structured self-critique to drive iterative quality improvements.

To use rubrics to ensure AI agent outputs meet quality thresholds, define your evaluation criteria and allow the system to automatically critique the generated content. It outputs JSON-structured critiques and repeats the refinement loop until your specific rubric is satisfied.

Yes, you can apply iterative testing and fixes to AI-generated scripts and algorithms. The system specifically supports code-like outputs by enforcing deterministic evaluation steps, applying structured self-critique, and repeating refinement loops until the code meets your defined criteria.

To audit iterative refinement loops for AI-generated documentation, rely on the system's built-in audit logging and JSON-structured critique outputs. These features track deterministic evaluation steps and provide clear justifications for all changes made during the self-critique process.

A limitation of using automated self-critique for software engineering workflows is that the iterative refinement process stops once a defined iteration limit is reached, even if the output has not fully satisfied the predefined quality criteria or rubric.