advanced-evaluation

Name: advanced-evaluation
Availability: InStock
Author: boraperusic

Automate LLM output evaluation with direct scoring and pairwise comparison.

Updated Apr 12, 2026

Search Tags:#bias-mitigation #llm-evaluation #pairwise-comparison #quality-assessment #prompt-evaluation #evaluation-rubrics #judgment-aggregation

Ready to run

npx skills add https://github.com/BoraPerusic/agents --skill advanced-evaluation-boraperusic

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: advanced-evaluation
Source: https://github.com/BoraPerusic/agents/tree/main/skills/to%20try/advanced-evaluation
Command: npx skills add https://github.com/BoraPerusic/agents --skill advanced-evaluation-boraperusic

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill provides structured methods for evaluating LLM outputs using LLMs as judges, enabling teams to design reliable evaluation pipelines, develop objective rubrics, and mitigate scoring biases.

Core Features & Use Cases

Direct Scoring and Pairwise Comparison strategies to evaluate model outputs.
Bias mitigation patterns (position bias, length bias, self-enhancement, verbosity, authority) and rubric-driven scoring.
Rubric generation and evaluation pipeline design for automation and reproducibility across experiments.
Example use cases include comparing model responses, building evaluation dashboards, and debugging evaluation systems across prompts and models.

Quick Start

Configure a two-model pairwise evaluation with swapped positions and apply a rubric to score responses.

Frequently Asked Questions about advanced-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I evaluate LLM outputs using LLMs as judges?▼

LLM evaluation uses LLMs as judges to produce consistent quality judgments through direct scoring and pairwise comparison. It automates evaluation pipelines by applying rubrics to score model responses and mitigate scoring biases.

What is the best way to compare multiple model responses automatically?▼

Pairwise comparison evaluates multiple model responses by using LLMs as judges to score them against each other. Swapping response positions and applying rubrics helps mitigate position bias and produces reproducible quality judgments.

How does bias mitigation work in LLM evaluation pipelines?▼

Bias mitigation in LLM evaluation applies structured patterns to counteract position bias, length bias, self-enhancement, verbosity, and authority bias. It uses rubric-driven scoring and swapped positions in pairwise comparisons to ensure objective judgments.

How do I generate evaluation rubrics for prompt evaluation?▼

Rubric generation creates objective scoring criteria for prompt evaluation and LLM output assessment. It designs rubric-driven scoring rules that enable automated, reproducible quality judgments across different models and prompts.

Can I use direct scoring to debug evaluation systems across different prompts?▼

Direct scoring evaluates individual LLM outputs against predefined rubrics to assess response quality. It supports debugging evaluation systems by providing consistent, reproducible quality judgments across various prompts and model responses.

When should I use pairwise comparison instead of direct scoring for quality assessment?▼

Pairwise comparison is ideal when comparing multiple model responses directly, while direct scoring suits individual output assessment. Both strategies use rubric-driven scoring and bias mitigation to produce consistent quality judgments.