skills.rest
agentclash
agentclashOFFICIAL
Explore all skills by this author →
GitHub

agentclash-eval-runner

Orchestrate AI agent evaluation runs against published challenge packs.

25|2|Updated Mar 4, 2026
Search Tags:#testing#cli#benchmarking#evaluation#regression#agentclash
One-click install
npx skills add https://github.com/agentclash/agentclash --skill agentclash-eval-runner
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agentclash-eval-runner
Source: https://github.com/agentclash/agentclash/tree/main/web/content/agent-skills/agentclash-eval-runner
Command: npx skills add https://github.com/agentclash/agentclash --skill agentclash-eval-runner

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the challenge of managing complex AI agent evaluations by providing a standardized interface to run, track, and inspect agent performance against specific challenge packs.

Core Features & Use Cases

  • Automated Eval Execution: Trigger runs across multiple agent deployments and input sets with precise control over scope and repetitions.
  • Live Monitoring & Inspection: Stream real-time events, view rankings, and diagnose failures through a unified CLI interface.
  • Use Case: A team needs to verify that a new agent deployment passes a regression suite before release; this Skill allows them to trigger the eval, follow the progress, and generate a scorecard to confirm performance metrics.

Quick Start

Use the agentclash-eval-runner skill to start a new evaluation run for the specified challenge pack and follow the live events until completion.

Frequently Asked Questions about agentclash-eval-runner

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run AI agent evaluations against specific challenge packs?▼

Trigger execution runs against published challenge packs using precise control over scope and repetitions. This runner orchestrates the process, requiring active workspace authentication and valid challenge pack identifiers to interface with the API.

What is the best way to monitor live AI agent evaluation events?▼

Stream real-time events, view rankings, and diagnose failures through a unified CLI interface. Live monitoring and inspection lets you track agent performance continuously during evaluation runs.

Can I compare multiple agent deployments during a single evaluation run?▼

Yes, you can trigger runs across multiple agent deployments and input sets. The runner facilitates complex workflows including multi-deployment comparisons to analyze relative performance metrics.

How do I test an AI agent regression suite before release?▼

Trigger the evaluation run for your regression suite, follow the live progress, and generate a scorecard to confirm performance metrics. This verifies new agent deployments pass required tests before release.

Do I need workspace authentication to execute agent benchmarking runs?▼

Yes, active workspace authentication is required to execute agent benchmarking runs. The runner needs valid credentials alongside valid challenge pack identifiers to interface with the AgentClash API.

Why does my AI agent evaluation run fail to start?▼

Evaluation runs fail to start without active workspace authentication or valid challenge pack identifiers. Ensure you provide correct API credentials and accurate challenge pack identifiers to interface successfully with the AgentClash API.

Related Skills

Explore More Software Engineering Skills→
agentclashagentclash
25

agentclash-challenge-pack-validation-publish

Validate and publish AgentClash challenge pack YAML configurations against workspace-backed schemas.

Official
Advanced
Sanchez-78Sanchez-78

deployment-readiness-review

Enforce a final deployment gate with cross-agent approvals and safety checks.

Community
Advanced
camoneartcamoneart
5

grafana-dashboards

Design and deploy Grafana dashboards for real-time system and application metrics.

Community
Intermediate
flutterflutter
2.8k

run-evals

Orchestrate automated evaluation of agent skills using isolated subagents and rubrics.

Official
Advanced