skills.rest
BoraPerusic
BoraPerusicCOMMUNITY
Explore all skills by this author →
GitHub

advanced-evaluation

Automate LLM output evaluation with direct scoring and pairwise comparison.

Updated Apr 12, 2026
Search Tags:#bias-mitigation#llm-evaluation#pairwise-comparison#quality-assessment#prompt-evaluation#evaluation-rubrics#judgment-aggregation
One-click install
npx skills add https://github.com/BoraPerusic/agents --skill advanced-evaluation-boraperusic
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: advanced-evaluation
Source: https://github.com/BoraPerusic/agents/tree/main/skills/to%20try/advanced-evaluation
Command: npx skills add https://github.com/BoraPerusic/agents --skill advanced-evaluation-boraperusic

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill provides structured methods for evaluating LLM outputs using LLMs as judges, enabling teams to design reliable evaluation pipelines, develop objective rubrics, and mitigate scoring biases.

Core Features & Use Cases

  • Direct Scoring and Pairwise Comparison strategies to evaluate model outputs.
  • Bias mitigation patterns (position bias, length bias, self-enhancement, verbosity, authority) and rubric-driven scoring.
  • Rubric generation and evaluation pipeline design for automation and reproducibility across experiments.
  • Example use cases include comparing model responses, building evaluation dashboards, and debugging evaluation systems across prompts and models.

Quick Start

Configure a two-model pairwise evaluation with swapped positions and apply a rubric to score responses.

Frequently Asked Questions about advanced-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate LLM outputs using LLMs as judges?▼

LLM evaluation uses LLMs as judges to produce consistent quality judgments through direct scoring and pairwise comparison. It automates evaluation pipelines by applying rubrics to score model responses and mitigate scoring biases.

What is the best way to compare multiple model responses automatically?▼

Pairwise comparison evaluates multiple model responses by using LLMs as judges to score them against each other. Swapping response positions and applying rubrics helps mitigate position bias and produces reproducible quality judgments.

How does bias mitigation work in LLM evaluation pipelines?▼

Bias mitigation in LLM evaluation applies structured patterns to counteract position bias, length bias, self-enhancement, verbosity, and authority bias. It uses rubric-driven scoring and swapped positions in pairwise comparisons to ensure objective judgments.

How do I generate evaluation rubrics for prompt evaluation?▼

Rubric generation creates objective scoring criteria for prompt evaluation and LLM output assessment. It designs rubric-driven scoring rules that enable automated, reproducible quality judgments across different models and prompts.

Can I use direct scoring to debug evaluation systems across different prompts?▼

Direct scoring evaluates individual LLM outputs against predefined rubrics to assess response quality. It supports debugging evaluation systems by providing consistent, reproducible quality judgments across various prompts and model responses.

When should I use pairwise comparison instead of direct scoring for quality assessment?▼

Pairwise comparison is ideal when comparing multiple model responses directly, while direct scoring suits individual output assessment. Both strategies use rubric-driven scoring and bias mitigation to produce consistent quality judgments.

Related Skills

Explore More Data & Analytics Skills→
apassuelloapassuello

llm-evaluation

Evaluate LLM outputs with automated metrics, human judgments, and benchmarking.

Community
Advanced
peterbamuhigirepeterbamuhigire
1

analytic-tradecraft

Apply ICD-203 standards and structured analytic techniques to produce auditable estimative judgments.

Community
Advanced
maragudkmaragudk
10

llm-as-a-judge

Build and validate LLM-as-Judge evaluators for automated quality assessment.

Official
Advanced
lucasFR2lucasFR2

advanced-evaluation

Automate LLM output evaluation with LLM-as-judge pipelines and bias mitigation.

Community
Advanced