meta-rubric-gen

Generate atomic evaluation rubrics by transforming a single LLM test-case input into mutually exclusive, testable YAML items that yield objective yes/no judgments. This ensures verifiable, MECE coverage for grading model responses.

Enforce safety constraints in LLM-as-a-judge rubrics by applying mandatory negative items for hallucinations and dangerous advice. The rubric generator allocates specific penalty budgets to prevent unsafe recommendations and policy violations.

Create MECE test-case scoring criteria in YAML format by emitting rubric lists with integer points and running structural checks. This validates mutually exclusive coverage and verifies penalty budget limits for single test-case LLM outputs.

You can use domain-specific references to build fact-first evaluation rubrics by incorporating agent-specific references and MCP tool capabilities. This enables context-aware checks for specialized domains like medical, coding, and legal.

The rubric generator supports automated checks for medical and legal LLM outputs by producing objective, test-case-level evaluation criteria. It applies to internal workflows across these domains to ensure safety-aware and verifiable grading.

LLM evaluation rubrics fail to prevent hallucinations when they lack explicit negative constraints. Generating atomic rubrics with 1-2 mandatory negative items and penalty budgets ensures hallucinations and policy violations are systematically penalized.