agentclash-challenge-pack-llm-judges

Configure LLM-as-judge scoring by defining YAML schema files that specify scorecard dimensions, judge limits, and evaluation modes. The skill validates these configurations within the AgentClash framework to enable rubric-based or comparative agent scoring.

Available evaluation modes include rubric-based scoring, assertion testing, reference comparison, and n-wise ranking. These modes support diverse AI testing needs, from checking specific assertions to comparing agent outputs against gold-standard reference answers.

Multi-model consensus increases confidence in judge results by enabling cross-model agreement checks. This mechanism compares evaluations across multiple LLM judges to ensure subjective scoring dimensions are reliably assessed.

Yes, assertion testing validates AI agent responses by defining specific conditions within YAML schema files. The skill processes these configurations through the AgentClash CLI to verify that agent outputs meet required deterministic or subjective criteria.

AgentClash supports n-wise ranking to evaluate complex agent outputs by comparing multiple responses against each other. This comparative scoring mode is configured through specific YAML schema definitions and executed via the CLI.

Use reference comparison when evaluating an agent response against a gold-standard answer, and use rubric scoring to assess subjective qualities like tone. The skill configures both modes through YAML schema for distinct AI testing needs.