skills.rest
guanyang
guanyangCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

advanced-evaluation

Implement LLM-as-a-Judge scoring with pairwise comparison and bias mitigation.

947|163|Updated Jan 13, 2026
Search Tags:#llm evaluation#llm-as-a-judge#bias mitigation#pairwise comparison#rubric generation#direct scoring#automated quality assessment
One-click install
npx skills add https://github.com/guanyang/antigravity-skills --skill advanced-evaluation-guanyang
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: advanced-evaluation
Source: https://github.com/guanyang/antigravity-skills/tree/main/skills/advanced-evaluation
Command: npx skills add https://github.com/guanyang/antigravity-skills --skill advanced-evaluation-guanyang

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of reliably evaluating LLM outputs by providing production-grade techniques for using LLMs as judges, mitigating common biases, and establishing clear evaluation rubrics.

Core Features & Use Cases

  • LLM-as-a-Judge Implementation: Learn and apply direct scoring and pairwise comparison methods.
  • Bias Mitigation: Understand and implement strategies for position bias, length bias, and more.
  • Rubric Creation: Design detailed rubrics for consistent and objective scoring.
  • Use Case: You are building an LLM application and need to compare the quality of different model responses to user prompts. This Skill helps you set up an automated evaluation pipeline to objectively score and rank these responses, ensuring high quality and consistency.

Quick Start

Use the advanced-evaluation skill to implement a pairwise comparison protocol for evaluating two model responses.

Frequently Asked Questions about advanced-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I use LLM-as-a-Judge for automated quality assessment of model outputs?▼

LLM-as-a-Judge automates quality assessment by using a model to evaluate outputs through direct scoring or pairwise comparison. You can build an automated pipeline to objectively score and rank responses, ensuring high quality and consistency.

What is the best way to mitigate position bias in pairwise comparison evaluation?▼

Mitigating position bias in pairwise comparison involves implementing strategies like position swapping. This technique alters the order of model responses during evaluation to prevent the judge from favoring the first or last presented answer.

How do I create evaluation rubrics for consistent direct scoring of LLM responses?▼

Creating evaluation rubrics for direct scoring involves designing detailed frameworks that establish clear, objective criteria. This ensures consistent and objective scoring across different model responses within an automated evaluation pipeline.

Do I need to understand evaluation taxonomies to use automated LLM evaluation systems?▼

Understanding evaluation taxonomies and metric selection is required for automated quality assessment. This knowledge ensures you correctly implement direct scoring, pairwise comparison, and bias mitigation techniques within your LLM application.

Does this approach address length bias when comparing different model responses?▼

Yes, addressing length bias is a core feature of robust LLM evaluation systems. The skill provides production-grade bias mitigation strategies to ensure longer responses are not unfairly favored over shorter, higher-quality answers.

Related Skills

Explore More Software Engineering Skills→
chicanoandres702chicanoandres702

llm-evaluation

Evaluate LLM outputs with automated metrics, human judgments, and A/B testing.

Community
Advanced
skeeper75skeeper75

moai-workflow-loop

Automate code quality feedback loops using LSP diagnostics and AST-grep security scans.

Community
Advanced
northseadlnorthseadl

advanced-evaluation

Evaluate LLM outputs using LLM judges with bias mitigation techniques.

Community
Advanced
JhabbigJhabbig

llm-evaluation

Design evaluation frameworks for LLM applications with automated and human scoring.

Community
Advanced