crisai-eval-baseline-campaign

What problem does it solve?

This Skill addresses the need for a comprehensive evaluation and acceptance baseline for crisAI's LLM product quality, encompassing routing accuracy, named-source resolution, source fit, grounding/citation, summary fidelity, artefact quality, policy gates, cost, latency, and human acceptance.

Core Features & Use Cases

• Evaluation Framework: Provides a structured framework for evaluating LLM product quality across multiple dimensions.
• Metrics Definition: Defines measurable metrics for each aspect of LLM performance.
• Baseline Creation: Helps create a baseline for product quality based on defined metrics.
• Use Case: For instance, when setting up an LLM product, this Skill can be used to ensure that the product meets the defined quality standards before release.

Quick Start

Load the skill with the command: uv run crisai run --skill crisai-eval-baseline-campaign