skills.rest
lluppesms
lluppesmsCOMMUNITY
Explore all skills by this author →
GitHub

agentic-eval

Evaluate and refine AI agent outputs through self-critique and reflection loops.

Updated Nov 21, 2025
Search Tags:#quality assurance#code generation#llm-as-judge#ai evaluation#self-critique#iterative refinement
One-click install
npx skills add https://github.com/lluppesms/dadabase.demo --skill agentic-eval-lluppesms
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agentic-eval
Source: https://github.com/lluppesms/dadabase.demo/tree/main/.github/skills/agentic-eval
Command: npx skills add https://github.com/lluppesms/dadabase.demo --skill agentic-eval-lluppesms

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill addresses the challenge of ensuring AI-generated outputs meet specific quality standards by enabling iterative self-improvement and evaluation.

Core Features & Use Cases

  • Iterative Refinement: Implements loops where an AI generates content, evaluates it against criteria, critiques it, and refines it.
  • Evaluation Strategies: Supports outcome-based, LLM-as-judge, and rubric-based evaluation methods.
  • Use Case: When generating code, this Skill can ensure the code not only functions but also passes associated unit tests through a feedback loop.

Quick Start

Use the agentic-eval skill to generate Python code for a given specification and then refine it based on unit test results.

Frequently Asked Questions about agentic-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I use self-critique to improve AI-generated code?▼

You can use self-critique to improve AI-generated code by implementing iterative refinement loops where the AI generates content, evaluates it against criteria, and refines it until it passes.

What is the LLM-as-judge evaluation method for AI agents?▼

LLM-as-judge is an evaluation strategy where an AI model evaluates its own outputs against specific rubrics or criteria, enabling automated quality assurance and iterative self-improvement.

How do I set up an evaluator-optimizer pipeline for quality assurance?▼

Set up an evaluator-optimizer pipeline by creating a loop that requires LLM calls for generation, critique, and refinement steps, applying rubric-based evaluation to ensure outputs meet quality standards.

Can I use iterative refinement to ensure generated code passes unit tests?▼

Yes, iterative refinement can ensure generated code passes unit tests by using a feedback loop where the AI generates code, runs tests, critiques failures, and refines the code accordingly.

What are the limitations of rubric-based evaluation for AI outputs?▼

Rubric-based evaluation requires multiple LLM calls for generation, critique, and refinement steps, which increases processing time and computational costs for each iterative loop.

Related Skills

Explore More Software Engineering Skills→
vchelaruvchelaru
11

agentic-eval

Coordinate generate-evaluate-critique-refine loops to improve AI agent outputs.

Community
Intermediate
PMDevSolutionsPMDevSolutions
16

wordpress-testing-workflows

Automate PHPUnit testing workflows for WordPress themes and plugins.

Official
Advanced
nanorepublicananorepublica
61

Testing Unit Tests

Standardize unit-test coding practices across Claude Code projects.

Community
Basic
Mentor-WizardMentor-Wizard
10

code-reviewer

Analyze code for security vulnerabilities and performance issues.

Official
Advanced