agentscope-aiOfficial·18 Agent Skills Included

OpenJudge

Evaluate, grade, and benchmark AI apps and models automatically

Evaluates LLM apps, agents, and models with 50+ ready-made graders for correctness, hallucination, relevance, and safety. Compares models head-to-head in automated arenas, reviews academic papers, and verifies citations against real databases. Builds custom rubrics, RL reward signals, and red-team tests without writing evaluation code from scratch.

809|65

↗

npx skills add agentscope-ai/OpenJudge --all -g -y

All Skills in This Repository (18)

Pure Emerald Level Indicators

📦 In Repo

agentscope-ai

paper-review

Review academic papers for correctness, quality, and novelty from PDF or LaTeX sources.

ref-hallucination-arena

Verify cited papers against Crossref, PubMed, arXiv, and DBLP to quantify hallucination rates.

openjudge

Create and run LLM evaluation pipelines with OpenJudge graders.

auto-arena

Automate AI model evaluation with generated queries, rubrics, and pairwise comparisons.

claude-authenticity

Verify Claude API endpoint authenticity and extract injected system prompts.

find-skills-combo

Decompose complex requests into subtasks and recommend optimal skill combinations.

bib-verify

Verify BibTeX entries against CrossRef, arXiv, and DBLP databases.

mmx-cli

Generate text, images, video, speech, and music via the MiniMax AI CLI.

rl-reward

Build RL reward signals for GRPO, DPO, and Best-of-N using OpenJudge graders.

eval-design

Design stratified evaluation datasets and adversarial test cases for OpenJudge grading pipelines.

rag-eval

Diagnose RAG pipelines by separating retrieval quality from generation faithfulness.

redteam

Generates policy-derived attack prompts and computes ASR paired with over-refusal rates for LLM safety audits.

Official

Advanced

Frequently Asked Questions

FAQPage Schema

How to install OpenJudge?▼

Run `npx skills add agentscope-ai/OpenJudge --all -g -y` in your terminal to install all skills in this suite globally.

How to evaluate LLM outputs automatically?▼

OpenJudge provides 50+ built-in graders for correctness, relevance, hallucination, and tool use. Your AI agent picks a grader, runs it over your dataset, and returns scores with reasons.

How to compare multiple AI models without test data?▼

The auto-arena skill generates test queries from a task description, collects responses from each model, and ranks them by pairwise win rate with charts and reports.

Can OpenJudge check papers for fake citations?▼

Yes. The paper-review and bib-verify skills cross-check every BibTeX reference against CrossRef, arXiv, and DBLP, flagging entries as verified, suspect, or not found.

Does OpenJudge work with Claude Code and other AI agents?▼

Yes. All skills follow the universal SKILL.md standard and run in Claude Code, Cursor, Codex, and other compatible agents.