advanced-evaluation

LLM evaluation uses structured patterns—direct scoring, pairwise comparison, and rubric generation—to turn subjective judgments into repeatable, auditable metrics. This Skill provides calibrated scoring with justification requirements, bias-mitigated comparisons, and domain-specific rubrics to standardize evaluations across responses.

Bias mitigation in LLM evaluation employs position swapping, anonymization, and multi-model evaluation patterns. This Skill implements these techniques within its pairwise comparison and direct scoring workflows to produce fairer, more reliable preference judgments.

Evaluation pipelines structure validation, scoring, and reporting workflows into auditable stages. This Skill provides reference implementations for direct scoring, pairwise comparison, and rubric-based assessment, enabling systematic benchmarking across tasks and domains with confidence-aware outputs.

Yes. Rubric generation produces domain-specific scoring rubrics that standardize evaluations. This Skill supports building custom rubrics tailored to your task requirements, combined with bias mitigation and confidence calibration for production-grade evaluation systems.

Direct scoring assigns criterion-based scores with explicit justification to individual responses; pairwise comparison evaluates two responses side-by-side to identify preferences. This Skill implements both, with bias mitigation built into comparisons and weighted scoring for direct assessments.

The Skill includes reference implementations and internal guidance on evaluation patterns, bias mitigation, and metrics design. Running scripts/evaluation_example.py demonstrates direct scoring, pairwise comparison, and rubric generation with sample prompts and responses to accelerate setup.