skills.rest
javiersgjavi
javiersgjaviCOMMUNITY
Explore all skills by this author →
Website𝕏@javsolgarGitHub

personal-agent-eval

Run reproducible LLM and agent benchmarks with deterministic checks and judge scores.

3|Updated Apr 16, 2026
Search Tags:#llm#evaluation#artifacts#benchmark#deterministic#fingerprints#openclaw
One-click install
npx skills add https://github.com/javiersgjavi/personal_agent_eval --skill personal-agent-eval
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: personal-agent-eval
Source: https://github.com/javiersgjavi/personal_agent_eval/tree/main
Command: npx skills add https://github.com/javiersgjavi/personal_agent_eval --skill personal-agent-eval

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

personal_agent_eval provides a reproducible framework to benchmark LLMs and autonomous agents, capturing runs, deterministic checks, and judge-driven scores with full audit trails.

Core Features & Use Cases

  • Run llm_probe campaigns against models via OpenRouter and record deterministic traces.
  • Benchmark OpenClaw-based agents with workspace diffs, artifacts, and judge results.
  • Inspect results, reuse fingerprints for incremental campaigns, and generate charts and reports.

Quick Start

Run a shipped pae campaign to start benchmarking immediately.

Frequently Asked Questions about personal-agent-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark LLMs and autonomous agents reproducibly?▼

To benchmark LLMs and autonomous agents reproducibly, run campaigns using configurable cases, suites, run profiles, and evaluation profiles that enforce deterministic checks and capture full audit trails.

How does fingerprint-based reuse work for LLM evaluation campaigns?▼

Fingerprint-based reuse for LLM evaluation matches previous run configurations to skip redundant executions, allowing incremental campaigns to reuse stored artifacts and deterministic traces from prior outputs.

Can I use an OpenRouter-backed judge for autonomous agent evaluation?▼

Yes, autonomous agent evaluation uses an OpenRouter-backed judge to score runs, combining deterministic checks with judge-driven results to generate comprehensive reports and workspace diffs.

Do I need OpenClaw agent workspaces to run benchmarks?▼

OpenClaw agent workspaces are required to benchmark OpenClaw-based agents, capturing workspace diffs, artifacts, and judge results, while standard LLM probes run directly via OpenRouter configurations.

What is the best way to inspect LLM benchmark results and generate reports?▼

The best way to inspect LLM benchmark results is to review stored outputs and artifacts under the audit directory, enabling you to generate charts, replay runs, and analyze deterministic traces.

Why does my LLM benchmark campaign fail deterministic checks?▼

LLM benchmark campaigns fail deterministic checks when configurations or run profiles introduce non-deterministic behavior, preventing fingerprint reuse and breaking the required reproducible audit trail.

Related Skills

Explore More Data & Analytics Skills→
Matymatyk-businessMatymatyk-business
61

run-deep-swe

Score AI models on the 113-task DeepSWE benchmark via OpenRouter.

Community
Advanced
krzemienskikrzemienski
2

performance-regression-detection

Detect performance regressions against baseline metrics above a 10% threshold.

Community
Advanced
ascorbicascorbic
120

onboarding

Automate Macrodata initial setup by creating identity, profile, and workspace state files.

Community
Advanced
NVIDIA-NeMoNVIDIA-NeMo
58

nemo-evaluator

Standardize LLM agent and RAG pipeline evaluation with composable metrics and reproducible benchmarks.

Official
Advanced