skills.rest
AlexYedi
AlexYediCOMMUNITY
Explore all skills by this author →
GitHub

judge-build

Scores build artifacts against a quality rubric using a cross-provider two-seat LLM quorum.

1|Updated Apr 9, 2026
Search Tags:#llm-as-judge#calibration#build artifacts#quality evaluation#cross-provider#quorum#rubric scoring
One-click install
npx skills add https://github.com/AlexYedi/Empire_State_Events_Pipeline_Take_3 --skill judge-build-alexyedi
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: judge-build
Source: https://github.com/AlexYedi/Empire_State_Events_Pipeline_Take_3/tree/main/.claude/skills/judge-build
Command: npx skills add https://github.com/AlexYedi/Empire_State_Events_Pipeline_Take_3 --skill judge-build-alexyedi

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Evaluating the quality of AI-generated build artifacts (skills, commands, hooks, references, code) with a single model is unreliable and biased toward its own family's work. This Skill provides a measurable, cross-provider quality signal by running two independent judges and merging their verdicts into one quorum decision. ## Core Features & Use Cases - Cross-provider quorum judging: Runs a Claude/Sonnet seat (house-aware) and a Gemini seat (independent), each scoring five rubric criteria with reasoning, then merges them into one verdict. - Mechanized pre-passes: Deterministically detects dangling file references and padding/density risks via shell hooks, enforcing score caps that models tend to under-apply. - Calibration loop: Prompts the user to agree or disagree with each verdict, logging the ack as calibration data; disagreements escalate interactively or fail-safe flag in autonomous mode. - Use Case: After generating a new skill or command file, run the judge to get per-criterion scores (correctness, completeness, convention adherence, anti-pattern avoidance, diagnostics) from both providers, with a merged verdict and run-log for audit. ## Quick Start Judge the build quality of the artifact at .claude/skills/trend-radar/SKILL.md using the cross-provider quorum in interactive mode.

Frequently Asked Questions about judge-build

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I judge the quality of an AI-generated skill or command file?▼

Provide the artifact path and optional spec, then the skill runs two independent judges — Claude/Sonnet and Gemini — against a five-criterion rubric. It merges their scores into one quorum verdict and asks you to agree or disagree for calibration.

What is a cross-provider LLM-as-judge quorum?▼

It is an evaluation setup where two models from different providers (Anthropic Claude and Google Gemini) independently score the same artifact against the same rubric. Agreement produces an automatic verdict; disagreement escalates to a human or triggers a fail-safe flag.

How are dangling references detected in build artifacts?▼

A shell hook (check-refs.sh) mechanically verifies that every load-bearing .claude/ path referenced in the artifact actually exists on disk. If any are missing, completeness is capped at 0.60 regardless of model opinion.

Does the judge automatically rewrite or block failing artifacts?▼

No. The skill only scores and flags; it never rewrites the artifact and never hard-blocks. In autonomous mode a disagreement produces a non-destructive fail-safe flag queued for later human review.

What happens when the Gemini judge fails or errors?▼

The adapter surfaces the error and exits non-zero rather than silently falling back to a single-judge pass. The Claude seat's result is recorded, the quorum is marked incomplete, and the run is flagged for re-run.

When should I not rely on this judge's verdict?▼

While the judge holds provisional-trust status, its scores should not hard-block work, especially on self-produced artifacts. The cross-provider quorum and prospective calibration runs are what retire the provisional label.

Related Skills

Explore More Software Engineering Skills→
Tx1207Tx1207
5

skill-quality-reviewer

Audit Claude Skills for YAML frontmatter, structure, and reference integrity.

Community
Advanced
azborgonovoazborgonovo

pareto

Ranks causes by outcome and spends a fifth of the effort on the highest-leverage interventions.

Community
Intermediate
wshobsonwshobson
39.3k

eval-harness-first

Build evaluation harnesses with goldens, graders, and baselines that gate fine-tuning runs.

Community
Advanced
luyzkkluyzkk

doubt-driven-development

Review a single decision through a bounded adversarial CLAIM-to-STOP process before committing.

Community
Advanced