AI Evaluation

Design evaluation frameworks for AI features with golden datasets and scoring rubrics.

1|1|Updated Feb 21, 2026
One-click install
npx skills add https://github.com/dtsong/claude-code-windows-setup --skill ai-evaluation-dtsong
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: AI Evaluation
Source: https://github.com/dtsong/claude-code-windows-setup/tree/main/skills/council/oracle/ai-evaluation
Command: npx skills add https://github.com/dtsong/claude-code-windows-setup --skill ai-evaluation-dtsong

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps you build robust frameworks to evaluate AI/LLM features, ensuring their quality, accuracy, and reliability through automated scoring and hallucination detection.

Core Features & Use Cases

  • Golden Dataset Creation: Develop high-quality datasets for rigorous AI testing.
  • Automated Scoring Rubrics: Design objective metrics to measure AI performance.
  • Hallucination Detection: Implement checks to identify and flag fabricated content.
  • Regression Testing: Build pipelines to catch performance degradation over time.
  • Use Case: You've developed a new AI feature for summarizing documents. Use this Skill to create a golden dataset of documents and their ideal summaries, then set up an automated scoring system to ensure new model versions don't degrade summary quality or introduce factual errors.

Quick Start

Design an AI evaluation framework for a new summarization feature, including golden dataset creation and automated scoring rubrics.

Frequently Asked Questions about AI Evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a golden dataset for LLM testing and automated scoring?

To create a golden dataset for LLM testing, you develop high-quality input and ideal output pairs, then define objective automated scoring rubrics to measure model performance against this version-controlled data.

What is the best way to detect hallucinations in AI features during development?

The best way to detect hallucinations in AI features is to implement comprehensive evaluation frameworks with automated checks that flag fabricated content against a golden dataset and objective scoring rubrics.

How do I build regression testing pipelines to catch LLM performance degradation?

You build regression testing pipelines by establishing CI/CD-compatible infrastructure that automatically scores new model versions against your golden dataset to catch performance degradation and factual errors.

Can I integrate AI evaluation frameworks with existing CI/CD pipelines?

Yes, you can establish CI/CD-compatible evaluation pipelines that automatically run automated scoring rubrics and hallucination detection checks to ensure new AI model versions do not degrade quality.

When do I need automated scoring rubrics for AI quality assurance?

You need automated scoring rubrics for AI quality assurance when you require objective metrics to measure LLM performance, detect hallucinations, and monitor regression across new model versions.

What evaluation dimensions should I define for AI summarization features?

For AI summarization features, you should define evaluation dimensions covering factual accuracy, hallucination detection, and summary quality, then measure them using a golden dataset and automated scoring rubrics.