loop

Name: loop
Availability: InStock
Author: Unclejobs

Run scored evaluations across prompt variants and promote winners meeting min_delta.

37|7|Updated Mar 19, 2026

Search Tags:#prompt optimization #benchmarking #evaluation harness #automation loop #score parsing #versioned promotion

Ready to run

npx skills add https://github.com/unclejobs-ai/second-claude-code --skill loop-unclejobs-ai

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: loop
Source: https://github.com/unclejobs-ai/second-claude-code/tree/main/skills/loop
Command: npx skills add https://github.com/unclejobs-ai/second-claude-code --skill loop-unclejobs-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

A loop without a benchmark produces random improvements, so this Skill helps you upgrade prompt assets only when they win against a fixed, numeric evaluation suite.

Core Features & Use Cases

Karpathy-style fixed-suite optimization: runs a candidate-variant loop where every change competes under the same time budget and scoring method.
Strict promotion gates: only promotes winners when numeric scores beat baseline by at least min_delta, with plateau and budget-based stopping.
Safe mutation scope: confines changes to an allowlisted target set and avoids prohibited files and directories during evaluation.

Quick Start

Run the loop for a selected suite name to benchmark baseline prompt assets, generate 3-5 safe variants, and promote the best winner only if the score improves by at least min_delta.

Frequently Asked Questions about loop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How does benchmark-driven prompt optimization work?▼

Benchmark-driven prompt optimization runs a fixed suite of scored evaluations across candidate variants, promoting only the winner if its numeric score beats the baseline by a minimum threshold.

How do I automate prompt evaluation across multiple variants?▼

You can automate prompt evaluation by running a fixed-suite loop that generates 3-5 safe variants, parses numeric scores, and retains elite candidates under a strict time budget.

When should I use a fixed evaluation suite for prompt assets?▼

Use a fixed evaluation suite when deterministic numeric scoring is required to prevent random improvements, ensuring every prompt change competes under the same scoring method and budget.

What are the promotion gates for safe prompt mutation?▼

Safe prompt mutation promotion gates require beating the baseline score by at least min_delta, while applying plateau and budget-based stopping criteria to prevent unnecessary iterations.

Can I restrict prompt evaluation changes to specific target files?▼

Yes, you can constrain changes to an allowlisted target set to ensure safe mutation scope, preventing the evaluation process from modifying prohibited files and directories.

How to stop prompt optimization loops from running indefinitely?▼

You can stop prompt optimization loops using hard stop gates including min_delta score thresholds, plateau detection, and budgeted iteration limits to halt runs when improvements stall.