skills.rest
aldy505
aldy505COMMUNITY
Explore all skills by this author →
WebsiteGitHub

advanced-evaluation

Implements LLM-as-a-Judge techniques for scoring, pairwise comparison, and bias mitigation.

Updated Feb 15, 2026
Search Tags:#llm evaluation#llm-as-a-judge#bias mitigation#pairwise comparison#rubric generation#direct scoring
One-click install
npx skills add https://github.com/aldy505/atrium --skill advanced-evaluation-aldy505
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: advanced-evaluation
Source: https://github.com/aldy505/atrium/tree/main/.agents/skills/context-engineering-collection/skills/advanced-evaluation
Command: npx skills add https://github.com/aldy505/atrium --skill advanced-evaluation-aldy505

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of reliably evaluating Large Language Model (LLM) outputs, enabling the creation of production-grade systems that mitigate bias and ensure consistent quality assessment.

Core Features & Use Cases

  • LLM-as-a-Judge Implementation: Provides patterns for using LLMs to score and compare other LLM outputs.
  • Bias Mitigation: Offers techniques to counteract common biases like position bias and length bias.
  • Rubric Generation: Guides the creation of detailed rubrics for consistent evaluation criteria.
  • Use Case: You are building a chatbot and need to evaluate which of two responses is more helpful and accurate. This Skill provides the framework to set up an LLM judge that can perform this comparison reliably, even swapping response order to avoid bias.

Quick Start

Use the advanced-evaluation skill to implement a pairwise comparison of two model responses, ensuring bias mitigation protocols are followed.

Frequently Asked Questions about advanced-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I use LLM-as-a-judge to evaluate model outputs reliably?▼

LLM-as-a-judge evaluates model outputs by using an LLM to score or compare responses. This skill provides patterns for direct scoring and pairwise comparison to ensure systematic, reliable quality assessment in production environments.

How do I mitigate position bias in LLM pairwise comparison?▼

Mitigate position bias in LLM pairwise comparison by swapping response order during evaluation. This skill implements bias mitigation protocols to counteract position, length, and self-enhancement biases for consistent results.

How do I generate rubrics for consistent LLM evaluation criteria?▼

Generate rubrics for consistent LLM evaluation criteria by following the structured guidance in this skill. It covers rubric creation techniques to establish detailed, standardized assessment benchmarks for automated quality scoring.

What is the best way to set up automated quality assessment for LLM development?▼

The best way to set up automated quality assessment for LLM development is implementing a structured evaluation pipeline. This skill offers pipeline design patterns for scalable, production-grade model output evaluation.

Can I perform direct scoring and pairwise comparison in the same evaluation pipeline?▼

Yes, you can perform direct scoring and pairwise comparison in the same evaluation pipeline. This skill covers both techniques alongside bias mitigation strategies, allowing you to build comprehensive, mixed-method assessment systems.

Related Skills

Explore More Software Engineering Skills→
wangyouanwangyouan

advanced-evaluation

Evaluate LLM outputs with direct scoring, pairwise comparison, and bias mitigation.

Community
Advanced
GSU-FrankJGSU-FrankJ

advanced-evaluation

Implement LLM-as-a-judge evaluation pipelines with bias mitigation and pairwise comparison.

Community
Advanced
viliawang-pmviliawang-pm

eval-harness-builder

Automate design and execution of systematic LLM application evaluations.

Community
Advanced
simplysmartaisimplysmartai

llm-evaluation

Evaluate LLM applications with automated metrics, human feedback, and LLM-as-judge patterns.

Community
Advanced