agentic-eval

Name: agentic-eval
Availability: InStock
Author: lzc0901

Evaluate and refine AI agent outputs through structured self-critique loops.

Updated Aug 23, 2026

Search Tags:#agents #rubric #refine #self-critique #evaluate #iterative-improvement #LLM-as-judge

Ready to run

npx skills add https://github.com/lzc0901/vibe_coding_template --skill agentic-eval-lzc0901

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: agentic-eval
Source: https://github.com/lzc0901/vibe_coding_template/tree/main/skills/ai-agents/agentic-eval
Command: npx skills add https://github.com/lzc0901/vibe_coding_template --skill agentic-eval-lzc0901

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Patterns and techniques for evaluating and improving AI agent outputs through structured self-critique and evaluation loops. This skill enables agents to assess their own results, iterate on quality, and apply rubric-based judgments to complex tasks.

Core Features & Use Cases

Self-critique and reflection loops to identify improvement opportunities.
Evaluator-Optimizer patterns separating generation and evaluation components for clarity and reliability.
Code-specific reflection and test-driven refinement workflows for software tasks.
Evaluation strategies including outcome-based assessment, LLM-as-judge comparisons, and rubric-based scoring.
Use Case: Improve AI-generated code, reports, and analyses by applying iterative quality control.

Quick Start

Define an evaluation rubric and a max iteration limit.
Implement generate(), evaluate(), and refine() components.
Run the loop until convergence and log the trajectory.

Frequently Asked Questions about agentic-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How does LLM-as-judge self-critique improve AI agent outputs?▼

LLM-as-judge self-critique improves AI agent outputs by running a generate-evaluate-refine cycle that applies rubric-based scoring to identify and correct weaknesses iteratively. This structured loop ensures high-stakes tasks like code generation meet convergence criteria before completion.

How do I set up an evaluator-optimizer pattern for LLM evaluation?▼

To set up an evaluator-optimizer pattern, define an evaluation rubric and a max iteration limit, then implement generate(), evaluate(), and refine() components. Run the loop until convergence is reached and log the trajectory for deterministic JSON outputs.

Can I use rubric-based scoring for data analysis and documentation tasks?▼

Yes, rubric-based scoring supports data analysis, documentation, and other high-stakes tasks beyond code generation. The modular generate-evaluate-refine cycle applies outcome-based assessment and LLM-as-judge comparisons to improve complex analytical results.

What are the limitations of iterative self-critique loops?▼

Limitations of iterative self-critique loops include dependency on configurable iteration limits and the quality of the evaluation rubric. If convergence criteria are not met within the max iteration limit, the loop terminates, potentially returning outputs that still require manual review.

Do I need external dependencies to run structured self-evaluation workflows?▼

No external dependencies are required to run structured self-evaluation workflows. The skill implements a modular generate-evaluate-refine cycle natively, producing deterministic JSON outputs and logging trajectories without needing additional component installations.