skills.rest
LauraFlorentin
LauraFlorentinCOMMUNITY
Explore all skills by this author →
GitHub

evaluation

Monitors agent performance and generates metrics for CI/CD pipelines and audits.

1|Updated Feb 11, 2026
Search Tags:#monitoring#latency#drift#cost#ab-testing#evaluate#agent-performance
One-click install
npx skills add https://github.com/LauraFlorentin/skills-marketplace --skill evaluation-lauraflorentin
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluation
Source: https://github.com/LauraFlorentin/skills-marketplace/tree/main/agentic-skills/skills/evaluation
Command: npx skills add https://github.com/LauraFlorentin/skills-marketplace --skill evaluation-lauraflorentin

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Systems for quantitatively and qualitatively measuring agent performance, reliability, and cost. Use when user asks to "evaluate agent performance", "benchmark my agent", "test agent quality", or mentions agent metrics, scoring, or performance assessment.

Core Features & Use Cases

  • Evaluation determines how well an agent performs (correctness, helpfulness, safety).
  • Monitoring determines how the system is running (latency, errors, cost) in a live environment.
  • Use cases include CI/CD quality gates, A/B testing comparisons, cost auditing, and drift detection to notice model behavior changes.

Quick Start

Run an evaluation suite against your test dataset and review the resulting metrics.

Frequently Asked Questions about evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate agent performance in a CI/CD pipeline?▼

You can evaluate agent performance by running evaluation routines over a test dataset within your CI/CD pipeline. This execution applies structured metric outputs and quality gates to measure correctness, helpfulness, and safety before deployment.

What is agent drift detection and when do I need it?▼

Agent drift detection is the process of monitoring for changes in model behavior within a live environment. You need it to automatically notice when shifting data or updates degrade correctness, helpfulness, or safety over time.

Can I use this to benchmark my agent for A/B testing?▼

Yes, you can benchmark your agent for A/B testing by running evaluation routines over test datasets for different agent versions. This generates structured metric outputs that compare correctness, helpfulness, and safety to identify the better variant.

How do I audit live latency and cost for my agent?▼

You audit live latency and cost by applying monitoring routines in your production environment. This tracks system runtime metrics like latency, errors, and operational costs, generating structured outputs for continuous performance assessment.

Does the evaluation support semantic scoring?▼

Yes, the evaluation supports optional semantic scoring alongside standard evaluation routines. This feature provides deeper qualitative measurement of agent outputs, calculating correctness, helpfulness, and safety metrics over your test datasets.

Related Skills

Explore More Data & Analytics Skills→
trueratruera
3.5k

trulens-evaluation-workflow

Orchestrates end-to-end TruLens evaluation workflows for LLM apps.

Official
Advanced
baphledbaphled
1

monitoring

Define Golden Signals metrics and configure alerts for system monitoring.

Community
Intermediate
danieldev24danieldev24

ci-cd-and-automation

Automates CI/CD pipeline setup and quality gates for code verification.

Community
Advanced
miptah21miptah21
2

data-observability

Implement data quality monitoring with Great Expectations, Soda, and SQL anomaly detection.

Community
Advanced