Evaluate AI-generated research outputs using structured JSON judgments.
npx skills add https://github.com/Butanium/claude-lab --skill research-judging
Please help me install this Agent Skill. Skill: research-judging Source: https://github.com/Butanium/claude-lab/tree/main/plugins/clab/skills/research-judging Command: npx skills add https://github.com/Butanium/claude-lab --skill research-judging
Evaluate AI-generated research outputs using structured JSON judgments.
Run claude -p on a directory of samples using the provided criteria and schema to generate structured judgments.
High-intent search queries and answers about installing and using this skill.
To evaluate AI-generated research outputs with a structured rubric, this Skill applies a defined criteria.md and a strict JSON schema to standardize scoring and enable auditable comparisons across multiple samples.
Structured JSON judging works by enforcing a defined rubric and a strict JSON schema across multiple AI research samples, ensuring consistent scoring and enabling machine-validated, auditable comparisons between outputs.
Yes, you can enforce a custom JSON schema defined in schema.json alongside a rubric in criteria.md to standardize scoring and ensure consistent, machine-validated results across hundreds of AI-generated drafts.
The best way to generate auditable comparisons across hundreds of research drafts is to run a model-agnostic evaluation pipeline that applies a consistent scoring rubric and outputs structured JSON judgments.
Yes, this research judging pipeline operates without external dependencies, relying solely on its internal criteria.md rubric and schema.json to produce structured JSON judgments for AI-generated research outputs.
You run automated research evaluation by executing the pipeline on a directory of samples using the provided criteria and schema, which generates structured JSON judgments with consistent scoring.
sarimofficialEvaluate hackathon submissions against project specs and judging criteria.
ken-cavanagh-gleanEvaluate LLM outputs with direct scoring, pairwise comparison, and rubric generation.
QingrongYExtract Theme, Problem, Innovation, Method, Baseline, and Dataset labels from one research paper into JSON.
vecearExtract typed JSON data from unstructured LLM outputs using Zod validation.
