Evaluate Rundale model quality with Sonnet 4.6 rubric scoring.
npx skills add https://github.com/dmooney/Rundale --skill rundale-bench
Please help me install this Agent Skill. Skill: rundale-bench Source: https://github.com/dmooney/Rundale/tree/main/.agents/skills/rundale-bench Command: npx skills add https://github.com/dmooney/Rundale --skill rundale-bench
It removes the guesswork of comparing and evaluating Rundale model outputs by providing repeatable, rubric-scored quality benchmarks with cost and performance reporting.
Run model-quality evaluation with: use the skill to benchmark a single target by saying: evaluate <model@base_url#env:VAR> on <provider>.
High-intent search queries and answers about installing and using this skill.
To benchmark LLM model quality end-to-end, this Skill runs targeted evaluation slices and performance probes, dispatching Sonnet 4.6 judge subagents to score outputs against a rubric and then ingesting finalized judgments with validations into a cost-aware report.
The best way to resume A/B testing after failures is using drain-queue mode, which scores existing on-disk judging bundles without re-running generation work. It supports re-dispatching missing or failed judge bundles before finalizing aggregates.
You can run a single-target A/B benchmark by specifying the target as <model@base_url#env:VAR> on <provider>. Bench mode then executes all relevant slices, including dialogue, intent, reaction, and simulation probes, before ingesting scored results.
Yes, you can evaluate dialogue-only outputs using eval-dialogue mode, which performs blind A/B/N testing focused exclusively on dialogue candidates. It scores across a 5-axis rubric and calculates per-candidate costs without executing the full performance probe suite.
If a provider has high error rates during model evaluation, the Skill flags these provider quirks and supports re-dispatching missing or failed judge bundles. This error-aware reporting ensures you can finalize aggregates without losing completed judging work.
alxxpersonalRoute tasks to the appropriate model and git strategy before dispatching subagents.
paulinghamScore candidate patches against a rubric using test outcomes and diff analysis.
dmooneyScore dialogue bundles against an 1820 rural Ireland authenticity rubric.
vnnklChains 9 planning and execution phases with context isolation in Claude Code.
