Design layered evaluation harnesses for long-running agent loops with deterministic gates.
npx skills add https://github.com/SollanSystems/loop-engineer --skill loop-evals
Please help me install this Agent Skill. Skill: loop-evals Source: https://github.com/SollanSystems/loop-engineer/tree/main/skills/loop-evals Command: npx skills add https://github.com/SollanSystems/loop-engineer --skill loop-evals
This Skill helps you design an evaluation harness that proves whether a long-running agent loop is actually done, instead of merely confident or self-reported as successful.
Ask the skill to design the evaluation harness for your loop, identify the required verification layers, and draft the repo-native metrics and regression structure.
High-intent search queries and answers about installing and using this skill.
To stop false completion in an agent loop, design an evaluation harness using a layered suite that treats false-completion-rate as a first-class metric. This enforces deterministic checks and trace-backed verification so loops cannot claim success without proof.
A layered evaluation harness for agent systems is a seven-layer suite that orders deterministic gates and regression checks before rubric-based judgment. This ensures deterministic verification remains authoritative over subjective model outputs.
Calibrate rubric judges for agent evaluation by setting up human-labeled calibration procedures alongside the evaluation suite. This maintains the trustworthiness of model judges over time by anchoring their assessments to verified human standards.
Prevent agents from gaming verification checks by implementing security probes, anti-cheat checks, and trace-based checks within the evaluation harness. These mechanisms catch shortcut behavior and verifier gaming across trace-backed iterations.
Yes, this evaluation approach requires building repo-native regression cases and metrics. These are integrated directly into the evaluation suite to provide deterministic gates and accurate proof-of-done verification for the agent loop.
gwpjpManages secure ML workflow automation with built-in guardrails and lifecycle controls.
estevanhernandez-stack-edDiagnose broken tests and distinguish harness breaks from test-logic failures.
Miosa-osaCalibrate LLM evaluators against human-labeled datasets and generate Markdown calibration reports.
JhabbigVerify work completion against fresh command output and execution evidence.
