evals-run

Coordinate Clawperator eval runs across local-dev and published runtimes.

2|Updated Feb 17, 2026
One-click install
npx skills add https://github.com/clawperator/clawperator --skill evals-run
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: evals-run
Source: https://github.com/clawperator/clawperator/tree/main/.agents/skills/evals-run
Command: npx skills add https://github.com/clawperator/clawperator --skill evals-run

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Coordinating and running Clawperator evals across local-dev and published runtimes to diagnose preflight failures, compare targets, and triage artifacts.

Core Features & Use Cases

  • Orchestrates eval runs for android-version tasks across multiple runtimes (local-dev and published) on emulators and devices.
  • Provides triage, replay, and rescore workflows with artifact outputs and run logs for debugging and validation.
  • Includes a built-in runbook and helper scripts to install and configure the operator and APKs on connected devices.

Quick Start

Run an android-version eval with a chosen agent, model, and runtime to validate the end-to-end workflow.

Frequently Asked Questions about evals-run

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I replay and rescore eval runs for Android emulator workflows?

You can replay and rescore eval runs by executing helper scripts that drive emulator and device workflows, generating artifact outputs and run logs to debug preflight failures and validate agent performance across runtime targets.

What is the best way to triage preflight failures from android-version eval runs?

Triage preflight failures by using the built-in runbook and helper scripts to orchestrate eval runs across local-dev and published runtimes, capturing artifact outputs and run logs to compare targets and validate agent behavior.

Can I run evals across multiple devices and emulator targets simultaneously?

Yes, you can run evals across multiple targets using the run_pack_a_android_version_matrix.sh script to drive emulator and device workflows, comparing runtime targets across connected devices for comprehensive end-to-end validation.

Do I need a specific operator setup to run android-version evals on connected devices?

Yes, you need to configure the operator setup using the references/evals-run.md runbook, which provides instructions to install and configure the operator and APKs on connected devices before executing the android-version eval workflows.

How does comparing local-dev and published runtimes help debug eval results?

Comparing local-dev and published runtimes isolates environment-specific issues by running the same android-version evals across both targets, exposing discrepancies in agent behavior and preflight failures through captured artifact logs.

Why are my android eval replay workflows not capturing artifacts correctly?

Replay workflows may fail to capture artifacts if the operator and APKs are not properly installed on connected devices, requiring a review of the runbook configuration and the run_android_version_eval.sh script execution environment.