darwinian-evolver

Automated prompt optimization is achieved by using an LLM-driven evolutionary search loop to iteratively test and refine prompt templates against a defined fitness function. This skill evaluates failure cases from training examples to evolve more effective extraction prompts.

An LLM-driven evolutionary search loop is an automated optimization method that iteratively mutates and evaluates artifacts like code snippets and SQL queries. It selects the fittest variations based on a custom-defined evaluator to progressively improve heuristic performance.

To optimize SQL queries and regex patterns, define custom organisms, evaluators, and mutators within a flexible framework. The evolutionary loop then applies these domain-specific fitness criteria to iteratively refine the patterns and queries for better performance.

Yes, this automated prompt engineering skill requires the OpenAI dependency for the LLM-driven evolutionary loop and Jinja2 for rendering prompt templates. You also need Python to execute the driver scripts and define custom problem templates.

Yes, you can define custom fitness functions for domain-specific code improvement by specifying custom evaluators and mutators. This allows the evolutionary search loop to iteratively refine artifacts based on your exact optimization criteria.

The main limitation of evolutionary search for automated prompt engineering is its reliance on measurable fitness functions. If your optimization criteria cannot be quantitatively evaluated against training examples, the mutator loop will lack the feedback needed to iteratively refine artifacts.