agentic-eval

You can evaluate and improve AI agent outputs by implementing iterative refinement loops that generate, critique, and refine responses using rubric-based evaluation and LLM-as-judge techniques. This skill provides patterns for structuring these evaluator-optimizer pipelines.

The LLM-as-judge approach uses an AI model to evaluate its own outputs against specific criteria. This skill facilitates this by providing rubric-based evaluation systems and self-critique mechanisms to identify weaknesses and enhance response quality.

You can set up test-driven code refinement by running generated code against tests, identifying failures through self-critique, and iteratively fixing errors. This skill provides the necessary patterns to automate this evaluator-optimizer pipeline.

Yes, you can use rubric-based evaluation for complex AI report generation. This skill allows the AI to review its own report against a defined rubric, identify weaknesses, and rewrite sections to improve clarity and accuracy.

Yes, AI self-critique requires structured output for reliable parsing of critique results. This skill emphasizes clear criteria and structured formatting to ensure effective evaluation and iterative refinement of agent responses.

Iterative refinement relies heavily on the clarity of your evaluation criteria; without clear rubrics, the self-critique mechanism may fail to identify actual weaknesses. This approach requires structured output formatting to reliably parse critique results.