Assess LLM outputs against defined metrics using DeepEval's pytest-based framework.
npx skills add https://github.com/SeanChenR/ghibli --skill deepeval-seanchenr
Please help me install this Agent Skill. Skill: deepeval Source: https://github.com/SeanChenR/ghibli/tree/main/.claude/skills/deepeval Command: npx skills add https://github.com/SeanChenR/ghibli --skill deepeval-seanchenr
DeepEval provides a structured, pytest-based framework for evaluating LLM outputs with 50+ metrics, enabling rigorous QA across RAG pipelines, chatbots, and agent workflows, and supporting multi-provider backends and component-level tracing.
Run deepeval with your test cases and metrics to begin automated, repeatable LLM evaluation.
High-intent search queries and answers about installing and using this skill.
You can evaluate LLM outputs using pytest by defining test cases and applying over 50 metrics to assess RAG pipelines, chatbots, and agent workflows, enabling automated and repeatable evaluations.
Over 50 evaluation metrics are available for RAG pipelines, conversational AI, agents, and safety tasks, supporting custom criteria to rigorously assess LLM outputs across various contexts.
Yes, component-level tracing is supported during LLM evaluation using the @observe decorator, allowing you to monitor and trace individual components within your RAG pipelines and agent workflows.
Yes, the framework supports multi-provider LLM backends and pluggable model providers, allowing you to configure and switch between different LLM providers for your evaluation tasks.
Creating a custom evaluation Skill requires YAML frontmatter with a name and description in SKILL.md, with optional resources stored under scripts, references, or assets directories.
zzxtbetaCreate and evaluate AI skills with SKILL.md and test cases.
ethan-rioOptimize LLM prompts using Arize trace data and evaluation scores.
teegin-gEvaluate and optimize LLM agent outputs using MLflow APIs.
Zhow01Assess and benchmark LLM outputs across automated metrics, human judgments, and safety checks.
