What problem does it solve? Setting up LLM-as-judge evaluation on Arize involves many moving parts — evaluator templates, classification choices, AI integrations, column mappings, tasks, and trigger-runs — and mistakes in any of them silently produce zero or wrong scores. This Skill guides the entire workflow so evaluations actually run and produce trustworthy labels. ## Core Features & Use Cases - Evaluator CRUD and versioning: Create, update, version, and delete LLM-as-judge evaluators with templates, classification choices, model selection, and invocation params via the ax CLI. - Task orchestration: Run evaluations as continuous monitors or one-time backfills on project spans, or score experiment runs on datasets, with column mappings derived from real exported data. - Run diagnostics: Troubleshoot cancelled runs, zero-span completions, eval index lag, and column mapping mismatches using a structured checklist. - Use Case: You want to detect hallucinations in production traces. The Skill samples recent spans, proposes a hallucination evaluator, creates it with an AI integration, maps template variables to actual span attributes, backfills 100 spans to validate, then enables continuous monitoring. ## Quick Start Ask the assistant to create a hallucination evaluator for your Arize project and run a small backfill to validate it before enabling continuous monitoring.
