nemo-evaluator

You can evaluate a RAG pipeline by defining a rubric for retrieval precision and response grounding, then running a reproducible benchmark that generates human-reviewable scores and reasoning artifacts.

An LLM-as-judge evaluation uses a language model to assess complex reasoning and agentic tool-calling validation, standardized through composable metrics and reproducible benchmark protocols.

Yes, the skill provides Bring Your Own Benchmark support that standardizes dataset versioning, judge-quality versus generator-quality experiments, and artifact generation for your custom data.

Yes, the framework explicitly separates generation-quality from judge-quality workflows, allowing you to independently measure LLM agent outputs and the performance of the evaluating judge model.

You create a rubric-driven evaluation by selecting from exact, numeric, LLM-as-judge, or RAG-specific composable metrics to match your specific rubric criteria for the agent task.

You need reproducible artifacts for LLM benchmarking when your workflow requires human-reviewable scores and reasoning traces to ensure transparent, domain-specific alignment and consistent model evaluation.