agentic-eval

LLM-as-judge self-critique improves AI agent outputs by running a generate-evaluate-refine cycle that applies rubric-based scoring to identify and correct weaknesses iteratively. This structured loop ensures high-stakes tasks like code generation meet convergence criteria before completion.

To set up an evaluator-optimizer pattern, define an evaluation rubric and a max iteration limit, then implement generate(), evaluate(), and refine() components. Run the loop until convergence is reached and log the trajectory for deterministic JSON outputs.

Yes, rubric-based scoring supports data analysis, documentation, and other high-stakes tasks beyond code generation. The modular generate-evaluate-refine cycle applies outcome-based assessment and LLM-as-judge comparisons to improve complex analytical results.

Limitations of iterative self-critique loops include dependency on configurable iteration limits and the quality of the evaluation rubric. If convergence criteria are not met within the max iteration limit, the loop terminates, potentially returning outputs that still require manual review.

No external dependencies are required to run structured self-evaluation workflows. The skill implements a modular generate-evaluate-refine cycle natively, producing deterministic JSON outputs and logging trajectories without needing additional component installations.