ce-optimize

Automate metric-driven optimization loops with controlled experiments and LLM judging.

Updated Apr 26, 2026
One-click install
npx skills add https://github.com/DieStok/ridder_lab_retreat_ai_hackathon_2026 --skill ce-optimize-diestok
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ce-optimize
Source: https://github.com/DieStok/ridder_lab_retreat_ai_hackathon_2026/tree/main/.agents/skills/compound-engineering/skills/ce-optimize
Command: npx skills add https://github.com/DieStok/ridder_lab_retreat_ai_hackathon_2026 --skill ce-optimize-diestok

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires git, python3, and includes scripts (resource) and references (resource) components.

What problem does it solve?

Automates metric-driven iterative optimization loops to converge on the best code/config variant by running controlled experiments and evaluating results against hard gates and/or LLM judge scores.

Core Features & Use Cases

  • Define a measurable optimization goal and a repeatable measurement harness.
  • Generate and manage hypothesis backlog, run experiments in isolation via worktrees, and log results.
  • Support hard metrics and LLM-based judging with stratified sampling and rubric-based scoring.
  • Automatic gating, baseline comparison, best-state tracking, and safe rollback to avoid regressions.
  • Applicable to tuning performance, clustering quality, search relevance, or any measurable output.

Quick Start

Define your optimization spec and run /ce-optimize to start the iterative experimentation loop.

Frequently Asked Questions about ce-optimize

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate metric-driven iterative optimization loops for code variants?

Metric-driven iterative optimization loops are automated by running controlled experiments in isolated git worktrees and evaluating results against hard gates or LLM judge scores. The process converges on the best code or config variant by automatically comparing results against a baseline and tracking the best state.

What is an LLM judge and how does it work for semantic quality evaluation?

An LLM judge evaluates semantic quality using rubric-based scoring and stratified sampling to select superior solutions. It acts as a measurement harness when a measurable objective requires human-like judgment rather than hard numerical metrics to assess output quality.

How do I run controlled experiments in isolation using git worktrees?

Running controlled experiments in isolation using git worktrees involves generating a hypothesis backlog and testing mutable files while keeping immutable files fixed. This safely manages experiment branches, logs results, and enables safe rollback to avoid regressions if a variant fails gating.

Do I need a defined measurement harness to use automated gating for optimization?

Yes, a clearly defined optimization goal and a repeatable measurement harness are required to use automated gating. The harness scores variants against hard gates or LLM judge criteria, enabling automatic baseline comparison, best-state tracking, and safe rollback to prevent regressions.

What's the best way to tune search relevance and clustering quality automatically?

Tuning search relevance and clustering quality automatically requires defining a measurable objective and running iterative optimization loops. By applying stratified sampling and rubric-based scoring, the system evaluates variants against hard gates to converge on the superior solution.

When should I not use metric-driven optimization loops for performance tuning?

You should not use metric-driven optimization loops when a project lacks a measurable objective, a repeatable measurement harness, or a clearly defined scope of mutable versus immutable files. Without these elements, the system cannot generate hypotheses, run controlled experiments, or gate results safely.