skills.rest
agentsope
agentsopeCOMMUNITY
Explore all skills by this author →
GitHub

agentsop-test-fix-loop

Run a verify-then-fix loop that converges on passing lint, type, and test results.

287|16|Updated May 20, 2026
Search Tags:#test automation#feedback loop#continuous integration#lint#llm coding#verify-then-fix#SWE-Bench
One-click install
npx skills add https://github.com/agentsope/SkillAlchemy --skill agentsop-test-fix-loop
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agentsop-test-fix-loop
Source: https://github.com/agentsope/SkillAlchemy/tree/main/skills/agentsop-test-fix-loop
Command: npx skills add https://github.com/agentsope/SkillAlchemy --skill agentsop-test-fix-loop

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps code-editing LLM agents reliably converge on “green” lint and test results by turning verifier output into the next prompt and enforcing loop termination.

Core Features & Use Cases

  • Verify-then-fix loop for coder agents: Runs a combined verifier command (lint/type/test) and feeds structured results back to the agent to drive the next edit.
  • Feedback distillation: Converts stdout/stderr/exit code into a concise, last-edit-anchored message (first failing signal, last traceback frame, ≤2k token target) to prevent context bloat.
  • Guardrails that prevent common failure modes: Caps iterations, detects stalls (same error repeating), commits each iteration for auditability, and escalates on environment/tooling failures instead of wasting agent turns.
  • Anti-cheat success detection: Verifies that “exit code 0” also means tests truly ran and were not weakened via skips/xfails/disabled tests.
  • Use cases: SWE-Bench-style patch loops, CI guardrails, and any “fix until tests pass” workflow where the success criterion is machine-verifiable.

Quick Start

Tell your AI agent to use the Test-Fix Loop policy so it edits, runs the verifier command, summarizes the first actionable failure as feedback, and keeps iterating until the verifier is truly green or the cap is reached.

Frequently Asked Questions about agentsop-test-fix-loop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate an LLM coding agent to fix errors until tests pass?▼

To automate an LLM coding agent to fix errors until tests pass, use a verify-then-fix loop that runs lint and test commands, captures the output, and feeds structured feedback back to the agent for the next edit.

How does feedback distillation work in a continuous integration test loop?▼

Feedback distillation in a continuous integration test loop converts stdout, stderr, and exit codes into a concise message anchored to the first failing signal and last traceback frame, targeting under 2k tokens to prevent context bloat.

Can I use a verify-then-fix protocol for SWE-Bench style patching?▼

Yes, you can use a verify-then-fix protocol for SWE-Bench-style patching, as it drives code-editing agents to converge on passing lint, type, and test results by applying machine-verifiable success criteria.

What is the best way to prevent an LLM coding agent from weakening tests to get a passing exit code?▼

The best way to prevent an LLM coding agent from weakening tests is to apply anti-cheat success detection, which verifies that exit code 0 means tests truly ran without being weakened via skips, xfails, or disabled tests.

Why does my automated code fix loop stall on the same lint or test errors?▼

An automated code fix loop stalls on the same lint or test errors when it lacks stall detection guardrails, which are needed to identify repeating errors, cap iterations, and escalate environment or tooling failures.

Do I need to commit code changes at every iteration of a test automation feedback loop?▼

Yes, you need to commit code changes at every iteration of a test automation feedback loop to ensure auditability, allowing each edit to be tracked as the agent progresses toward a green verifier state.

Related Skills

Explore More Software Engineering Skills→
mkalkeremkalkere
4

bash

Executes shell commands in the workspace with output and error capture.

Community
Basic
dgethingsdgethings

pyats-routing

Audit OSPF, BGP, EIGRP, and IS-IS routing state and convergence with pyATS.

Community
Advanced
DicklesworthstoneDicklesworthstone
70

flywheel-discord

Enforce strict isolation rules for public Discord assistant interactions.

Community
Basic
bennybennisonbennybennison

autonomous-loops

Execute autonomous agent loops with stop conditions and iteration caps.

Community
Advanced