Evaluate and refine AI agent outputs through structured critique loops.
npx skills add https://github.com/eggboy/skills --skill agentic-eval-eggboy
Please help me install this Agent Skill. Skill: agentic-eval Source: https://github.com/eggboy/skills/tree/main/agentic-eval Command: npx skills add https://github.com/eggboy/skills --skill agentic-eval-eggboy
AI agents often produce outputs that require structured evaluation and guided refinement to reach acceptable quality. This skill provides patterns for self-critique, evaluation-then-improvement loops, and separate evaluator-optimizer workflows to enable reliable, incremental improvements.
Provide an initial task to the agent and allow it to iterate evaluation and refinement until it meets criteria.
High-intent search queries and answers about installing and using this skill.
An evaluator-optimizer pipeline separates generation from evaluation, enabling an LLM judge to score outputs and apply targeted refinements. This pattern uses rubric-based scoring and structured JSON feedback to ensure reliable, incremental improvements.
A self-critique loop is a pattern where AI agents assess and critique their own outputs to generate targeted improvements. You need it when agents produce outputs requiring structured evaluation and guided refinement to reach acceptable quality.
Test-driven refinement applies code-specific reflection to evaluate generated code against test criteria. The agent iteratively critiques outputs using structured JSON feedback and fixed iteration limits until meeting the required test outcomes.
Yes, you can configure rubric-based scoring and outcome-focused assessment with fixed iteration limits. This strategic evaluation method ensures reliability, safety, and traceability while preventing infinite critique loops during the refinement process.
Limitations of self-critique loops include the need for configurable prompts and fixed iteration limits to prevent infinite cycles. Without JSON-structured feedback and clear evaluation criteria, the iterative improvement process may lack reliability and traceability.
githubAutomate iterative evaluation and refinement of AI agent outputs with JSON feedback.
mauriciodelrioConfigure logging, monitoring, and alerting for cloud applications with AWS CloudWatch and Sentry.
lucasgaldinosAutomate agent output evaluation with rubric-based scoring and self-critique.
Teased-oChroid-orrAImplement iterative refinement and evaluation frameworks for AI agent outputs.
