empirical-prompt-tuning

Run iterative empirical loops to evaluate and refine agent prompts.

2|Updated Feb 8, 2026

Search Tags:#metrics #safety #iteration #subagent #prompt-engineering #empirical-evaluation

Ready to run

npx skills add https://github.com/genki-sano/dotfiles --skill empirical-prompt-tuning-genki-sano

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: empirical-prompt-tuning
Source: https://github.com/genki-sano/dotfiles/tree/main/.claude/skills/empirical-prompt-tuning
Command: npx skills add https://github.com/genki-sano/dotfiles --skill empirical-prompt-tuning-genki-sano

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill provides a structured approach to evaluating and improving prompts by running empirical loops with subagents, surfacing ambiguities, and iterating until performance stabilizes.

Core Features & Use Cases

Iterative evaluation workflow: dispatch subagents to test prompts, collect self-reports and tool usage metrics, and guide improvements.
Ambiguity detection and correction: identify unclear instructions and fill gaps with precise refinements.
Safe, repeatable governance: enforce evaluation boundaries, hold-out scenarios, and reproducible scoring.

Quick Start

Begin the empirical evaluation loop by dispatching a subagent to read the target prompt, run scenarios, and report results for iteration.

Frequently Asked Questions about empirical-prompt-tuning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I empirically evaluate and refine agent prompts?▼

Empirically refine agent prompts by running iterative evaluation loops with isolated subagents that test scenarios, collect metrics, and surface ambiguities until performance stabilizes. The process dispatches subagents to execute the target prompt, measure results against scoring rules, and report findings for the next iteration.

What is empirical prompt tuning for AI agents?▼

Empirical prompt tuning is a structured approach to improving AI agent instructions by running automated testing loops. It dispatches subagents to execute target prompts across scenarios, collects self-reports and tool usage metrics, and guides iterative refinements until performance metrics converge.

How do I test prompt safety and prevent prompt injection during evaluation?▼

Test prompt safety during evaluation by using isolated subagents to run scenarios, preventing prompt injection and data leakage. The skill enforces safe execution boundaries, hold-out scenarios, and reproducible scoring to ensure secure and repeatable prompt testing.

Can I use this empirical evaluation method for slash commands and task instructions?▼

Yes, empirical evaluation applies to skill prompts, slash commands, and task instructions across agents. The skill automates iterative testing and improvement for any agent prompt by applying clear scoring rules, thresholds, and reporting formats until metrics converge.

What's the best way to detect ambiguities in LLM prompt instructions?▼

Detect ambiguities in LLM prompts by dispatching subagents to run scenarios and surface unclear instructions. The empirical evaluation loop identifies gaps in the prompt, collects performance metrics, and guides precise refinements to fill those gaps until the prompt stabilizes.

Why do my agent prompts fail to converge during iterative testing?▼

Agent prompts fail to converge during iterative testing when scoring rules lack clear thresholds or evaluation boundaries are inconsistent. The skill enforces reproducible scoring, hold-out scenarios, and safe isolated subagents to ensure metrics stabilize across repeated empirical loops.