Score AI agent responses against standardized rubrics with weighted totals.
npx skills add https://github.com/slabgorb/sidequest --skill judge-slabgorb
Please help me install this Agent Skill. Skill: judge Source: https://github.com/slabgorb/sidequest/tree/main/.pennyfarthing/skills/pf-judge Command: npx skills add https://github.com/slabgorb/sidequest --skill judge-slabgorb
The Judge Skill provides a standardized framework to evaluate AI responses, ensuring objective, repeatable benchmarking across tasks and teams.
Run an evaluation scenario by supplying contestant specs, a challenge description, and a sample response to generate scores and a structured evaluation narrative.
High-intent search queries and answers about installing and using this skill.
To evaluate AI agent responses against standardized rubrics, supply contestant specs, a challenge description, and a sample response to generate scores and a structured evaluation narrative. This enforces consistent dimensions like correctness, depth, quality, and persona for objective benchmarking.
The best way to benchmark AI responses for reproducibility is using a framework that outputs deterministic weighted totals, rubric anchors, and detailed per-mode breakdowns. This ensures repeatable and auditable evaluation across tasks and teams.
You can compare two AI agents side-by-side using the comparison evaluation mode, which scores responses against shared rubric dimensions. It generates weighted totals and detailed breakdowns to objectively highlight performance differences.
When evaluating prompt-engineering outputs, you should score the dimensions of correctness, depth, quality, and persona. These standardized dimensions ensure comprehensive objective evaluation across benchmark and research workflows.
The AI evaluation framework supports phase-based testing for experiments through its phase evaluation mode. This mode facilitates facilitator-mode evaluations and product tests, generating structured narratives and weighted totals across multiple testing stages.
manuelbrandner85Create, evaluate, and refine AI skills with automated testing and benchmarking.
sourcegraphScore benchmark tasks on clarity, verifier quality, and reproducibility with Python scripts.
monch1962Design ClickHouse schemas and optimize analytical queries for large datasets.
sanjogkadayat-webEvaluate AI-generated Python code pipelines across correctness, formatting, performance, and reliability.
