judge-quality

Quantify output quality issues using a deterministic scoring schema.

Updated Jan 16, 2026

Search Tags:#quality #alignment #scoring #evaluate #AI-outputs #naturalness #richness

Ready to run

npx skills add https://github.com/HsunGong/prep --skill judge-quality

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: judge-quality
Source: https://github.com/HsunGong/prep/tree/main/.github/skills/validate
Command: npx skills add https://github.com/HsunGong/prep --skill judge-quality

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a structured approach to validating AI-generated outputs at Stage 4, enabling consistent scoring of quality dimensions such as naturalness, alignment, and richness.

Core Features & Use Cases

Stage-4 Scoring Framework: Quantifies naturalness, factuality, alignment, and content richness.
Reproducible Evaluation: Provides a standardized workflow that can be applied across tasks and domains.
Use Case: QA teams evaluating model outputs during product development, content generation, or safety reviews.

Quick Start

To use this Skill, provide the model's latest response to the judge and review the resulting scores. Then export results for reporting and feedback loops.

Frequently Asked Questions about judge-quality

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I evaluate AI output quality across naturalness, alignment, and richness?▼

To evaluate AI output quality across naturalness, alignment, and richness, you apply a structured Stage 4 scoring framework. This process uses a deterministic rubric and a lightweight runner to quantify output issues and generate structured results.

What is Stage 4 validation for generated AI responses?▼

Stage 4 validation is a standardized scoring process that quantifies naturalness, factuality, alignment, and content richness in generated responses. It enables reproducible evaluation across creative, factual, and aligned tasks to ensure output consistency.

How do I score model outputs reproducibly across different domains?▼

You can score model outputs reproducibly across different domains by applying a simple rubric and a lightweight runner like sft_judge_quality.py. This workflow produces structured results that standardize evaluation across tasks and iterations.

Can I use a deterministic scoring schema for QA teams reviewing model outputs?▼

Yes, QA teams can use a deterministic scoring schema to review model outputs during product development, content generation, or safety reviews. The schema provides a standardized workflow that quantifies quality dimensions and identifies output issues.

Do I need a specific runner to quantify alignment and factuality issues?▼

You need a lightweight runner such as sft_judge_quality.py along with a simple rubric to quantify alignment and factuality issues. Providing the model's latest response to this setup produces structured scores for reporting and feedback loops.