eval-grading-quality-audit

Audit grading quality by deploying sub-agents that independently read the rubric, extracted student answers, and grading outputs to score each question on five criteria and report root causes. This automated per-question review ensures consistency with awarded marks across a target cohort.

Per-question grading quality assurance uses LLM sub-agents to independently re-judge student work against a reference rubric and verify consistency with awarded marks. It enforces evidence cross-check and rejects heuristic scoring to provide traceable diagnostics and audit commentary.

Generate a grading quality audit report by orchestrating independent judgments per student, scoring each question, and aggregating results into grading_quality_audit.json. This file includes per-question failure_stage, root_cause_type, and a student-level summary for remediation planning.

You can use this grading audit across multiple students and years by providing the rubric, extracted student answers, and grading outputs for the target cohort. The system orchestrates independent sub-agent judgments per student and aggregates the results into a consolidated JSON file.

The grading audit enforces no heuristic scoring and requires evidence cross-check against the reference rubric and failure taxonomy. This ensures all per-question diagnostics and root-cause diagnoses are fully traceable and backed by verified grading evidence.

To start a grading quality audit, you need to provide the reference rubric, extracted student answers, and grading outputs for your target cohort. These inputs allow the sub-agents to launch per-student audits and generate the aggregated grading_quality_audit.json file.