Iteratively evaluate and refine AI agent outputs against scoring thresholds.
npx skills add https://github.com/rhyme17/NexusAI --skill agentic-eval-rhyme17
Please help me install this Agent Skill. Skill: agentic-eval Source: https://github.com/rhyme17/NexusAI/tree/main/.github/skills/agentic-eval Command: npx skills add https://github.com/rhyme17/NexusAI --skill agentic-eval-rhyme17
It helps you reduce low-quality or unreliable AI agent outputs by adding an evaluation-and-refinement loop, so results converge toward accuracy and compliance with clear criteria.
Ask an AI agent to generate an answer for a quality-critical task, then evaluate that answer against your rubric and iteratively refine it up to three rounds until it meets a minimum score threshold.
High-intent search queries and answers about installing and using this skill.
You can iteratively improve AI agent outputs by applying a generate, evaluate, critique, and optimize loop that scores responses against your rubric dimensions and refines failures until meeting a minimum score threshold.
An LLM-as-judge evaluation pipeline generates code and tests, runs them, then uses structured PASS/FAIL JSON critiques to iteratively apply fixes until tests pass or the bounded iteration limit is reached.
Yes, self-critique reflection loops support quality assurance for report writing by generating structured JSON critiques of analytical reasoning outputs and refining only the failed dimensions to ensure compliance with explicit success criteria.
To prevent infinite loops during AI agent evaluation and refinement, enforce best-practice guardrails such as maximum iteration limits, convergence checks, and retain iteration history for debugging and analysis.
Iterative refinement supports analytical reasoning tasks by applying rubric-based scoring with weighted dimensions, using outcome checks and LLM-as-judge comparisons to drive the optimizer pipeline toward quality-critical responses.
The primary limitation of LLM-as-judge for iterative code refinement is the bounded iteration limit, meaning the loop stops after a set number of retries or convergence checks even if tests still fail.
selfagencyEvaluate and iteratively refine AI agent outputs using structured rubrics.
miptah21Run structured evaluation and iterative refinement loops for AI agent outputs.
batihandevRun Unity EditMode and PlayMode tests and parse TestResults XML reports.
jaimedhenriquesEvaluate and iteratively refine AI outputs using structured JSON feedback.
