Build and run Phoenix evaluators for AI/LLM applications.
npx skills add https://github.com/Arize-ai/phoenix --skill phoenix-evals
Please help me install this Agent Skill. Skill: phoenix-evals Source: https://github.com/Arize-ai/phoenix/tree/main/skills/phoenix-evals Command: npx skills add https://github.com/Arize-ai/phoenix --skill phoenix-evals
Build and run evaluators for AI/LLM applications using Phoenix. This Skill provides a framework to create code-based evaluators, LLM evaluators, and calibration workflows to ensure reliable AI system evaluation.
Install and configure the Phoenix evaluation toolkit, then create a simple evaluator (code or LLM), and run it against a small dataset to validate behavior.
High-intent search queries and answers about installing and using this skill.
To evaluate LLM application outputs using Phoenix, you can build code-based evaluators for deterministic checks or LLM-based judges for subjective quality, then run them through experiment pipelines against your datasets.
Yes, you can build custom code-based evaluators for regex and JSON validation. The framework supports deterministic checks including exact matches, regex patterns, and JSON schema validation for AI outputs.
Code-based evaluators perform deterministic checks like regex and exact matches, while LLM-based judges handle subjective quality judgments. LLM judges require human calibration to ensure reliable evaluation results.
You run evaluation experiments across datasets and tasks by configuring end-to-end experiment workflows. These pipelines execute your datasets, apply tasks, and run evaluators to generate results using Phoenix libraries.
Yes, the Phoenix evaluation toolkit works with both Python and TypeScript. It provides SDKs for both environments, allowing you to create evaluators and run experiment workflows in your preferred language.
You calibrate LLM evaluators to match human judgments through calibration workflows. This process aligns subjective LLM-based quality assessments with human evaluation standards for reliable AI system scoring.
akillnessTrace, evaluate, and manage prompts for LLM applications via LangSmith SDKs.
maragudkCrafts, reviews, and refines prompts for LLM pipelines, including system and LLM-as-Judge prompts.
zhuangbiaoweiTrace, evaluate, and monitor LLM applications with OpenTelemetry integration.
Arize-aiCapture AI application traces, evaluations, and prompts with OpenTelemetry.
