advanced-evaluation

Name: advanced-evaluation
Availability: InStock
Author: guanyang

Implement LLM-as-a-Judge scoring with pairwise comparison and bias mitigation.

947|163|Updated Jan 13, 2026

Search Tags:#llm evaluation #llm-as-a-judge #bias mitigation #pairwise comparison #rubric generation #direct scoring #automated quality assessment

Ready to run

npx skills add https://github.com/guanyang/antigravity-skills --skill advanced-evaluation-guanyang

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: advanced-evaluation
Source: https://github.com/guanyang/antigravity-skills/tree/main/skills/advanced-evaluation
Command: npx skills add https://github.com/guanyang/antigravity-skills --skill advanced-evaluation-guanyang

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of reliably evaluating LLM outputs by providing production-grade techniques for using LLMs as judges, mitigating common biases, and establishing clear evaluation rubrics.

Core Features & Use Cases

LLM-as-a-Judge Implementation: Learn and apply direct scoring and pairwise comparison methods.
Bias Mitigation: Understand and implement strategies for position bias, length bias, and more.
Rubric Creation: Design detailed rubrics for consistent and objective scoring.
Use Case: You are building an LLM application and need to compare the quality of different model responses to user prompts. This Skill helps you set up an automated evaluation pipeline to objectively score and rank these responses, ensuring high quality and consistency.

Quick Start

Use the advanced-evaluation skill to implement a pairwise comparison protocol for evaluating two model responses.

Frequently Asked Questions about advanced-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I use LLM-as-a-Judge for automated quality assessment of model outputs?▼

LLM-as-a-Judge automates quality assessment by using a model to evaluate outputs through direct scoring or pairwise comparison. You can build an automated pipeline to objectively score and rank responses, ensuring high quality and consistency.

What is the best way to mitigate position bias in pairwise comparison evaluation?▼

Mitigating position bias in pairwise comparison involves implementing strategies like position swapping. This technique alters the order of model responses during evaluation to prevent the judge from favoring the first or last presented answer.

How do I create evaluation rubrics for consistent direct scoring of LLM responses?▼

Creating evaluation rubrics for direct scoring involves designing detailed frameworks that establish clear, objective criteria. This ensures consistent and objective scoring across different model responses within an automated evaluation pipeline.

Do I need to understand evaluation taxonomies to use automated LLM evaluation systems?▼

Understanding evaluation taxonomies and metric selection is required for automated quality assessment. This knowledge ensures you correctly implement direct scoring, pairwise comparison, and bias mitigation techniques within your LLM application.

Does this approach address length bias when comparing different model responses?▼

Yes, addressing length bias is a core feature of robust LLM evaluation systems. The skill provides production-grade bias mitigation strategies to ensure longer responses are not unfairly favored over shorter, higher-quality answers.