continuous-eval
OfficialAutomate quality and safety assessments post-deployment with Ragas-based metrics.
Authoraws-samples
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill automates post-deployment quality and safety assessments of AI agents using Ragas-based metrics, ensuring reliability in AI workflows.
Core Features & Use Cases
- Post-Deployment Evaluation: Performs evaluations immediately after deployment and at hourly intervals.
- Ragas Metrics: Uses faithfulness, answer relevance, context precision, toxicity, and PII leakage metrics.
- Golden Dataset Management: Manages golden datasets for regression gates and provides feedback on agent performance.
- Use Case: After deploying a new AI agent, use this Skill to evaluate its performance against a golden dataset and regression gate criteria.
Quick Start
Run the continuous-eval skill on the latest deployment version with the command: continuous-eval target:version --mode canary
Dependency Matrix
Required Modules
ragaslangfusepresidio-analyzerdetoxifyawslabs.prometheus-mcp-server
Components
scriptsreferencesassets
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: continuous-eval Download link: https://github.com/aws-samples/sample-oh-my-aidlcops/archive/main.zip#continuous-eval Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.