agent-validation-loop

Validates developer tools by running disposable subagents through closed-loop scenarios and fixing reported friction.

22|1|Updated Mar 30, 2026
One-click install
npx skills add https://github.com/mizchi/vlmkit --skill agent-validation-loop-mizchi
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: agent-validation-loop
Source: https://github.com/mizchi/vlmkit/tree/main/.apm/skills/vlmkit/workflows/agent-validation-loop
Command: npx skills add https://github.com/mizchi/vlmkit --skill agent-validation-loop-mizchi

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Unit tests and code review cannot tell you whether a fresh agent can actually use your CLI, library, or agent harness. This Skill closes that gap by running disposable subagents against a measurable scenario, treating their friction as the specification, and iterating until the tool's signal quality converges. ## Core Features & Use Cases - Closed-loop subagent validation: Spawn fresh agents with explicit read/forbidden file lists, a round budget, and a deterministic success criterion, then compare convergence across tool versions. - Friction-driven fixing: File one issue per agent complaint, commit per friction point with the agent's verbatim quote, and write versioned validation reports under docs/reports/. - Convergence tracking and stopping criteria: Track per-agent metrics in a comparison table, tighten round budgets as the tool matures, and stop when diminishing returns appear. - Use Case: You built a CLI whose output agents must parse. Run this loop: agent-a reports the notation is ambiguous, you fix it in one commit, agent-b confirms convergence improved from 10.3% to 0.2% diff. ## Quick Start Ask the agent to run the agent-validation-loop on your tool by defining a fixture with a brief and goal artifacts, then spawning a fresh subagent to attempt the task.

Frequently Asked Questions about agent-validation-loop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test whether an AI agent can use my CLI tool?

Run a closed-loop validation: give a fresh subagent a brief, goal artifacts, and your tool's docs, with a deterministic success criterion and a round budget. Treat the agent's quoted friction as the spec, fix one issue per commit, and re-run a new agent on the same scenario.

What is a closed-loop validation scenario for agent tools?

It is a fixture containing a brief, goal artifacts, and per-agent attempt directories, where success is measured by a deterministic criterion like a target diff percentage or expected output. Without a measurable outcome, validation degrades into subjective impressions.

When should I not use subagent-based tool validation?

Skip it for pure algorithmic correctness, where a unit test is cheaper, and for one-off scripts with no UX surface. It is also wasteful when you already know what to fix, since the subagent overhead only pays off when the gap between working code and user success is unknown.

Why must validation subagents be forbidden from reading prior attempts?

Agents that read prior attempts converge faster but learn nothing about the tool, because they are memorizing rather than using it. The prompt must name each forbidden path explicitly, including the goal's source code and earlier agents' attempt directories.

How do I know when to stop an agent validation loop?

Stop when each new run surfaces fewer gaps, remaining variance comes from agent-side initial state rather than tool signal quality, and per-fix commits shrink to small annotations. Typically after 7-9 runs, average improvement per fix drops below one percentage point.