meta-learning-evolution

To automate meta-learning pipeline design, you evolve code, prompts, and configurations using a two-level workflow where a genetic algorithm handles outer-loop optimization and an LLM invents new primitives when search stagnates. This requires a deterministic evaluation harness and experiment templates.

The best way to evolve loss functions and hyperparameters automatically is using Level-1 GA-guided evolution for outer-loop optimization. This approach systematically explores and mutates configurations to discover optimal parameters across GA and LLM-driven workflows.

You should use LLM-driven prompt discovery instead of genetic algorithms when your GA search stagnates. Level-2 LLM-driven evolution supplements GA search by injecting new capabilities and inventing new primitives to unblock the optimization process.

Yes, you need a deterministic evaluation harness to set up two-level evolution workflows. A deterministic harness ensures reproducible experiment templates, providing explicit evaluation metrics to reliably measure the performance of evolved prompts, code, and configurations.

Yes, you can use this approach to optimize reward functions for automated development tasks. The workflow applies to research and development tasks involving the automated exploration of loss functions, rewards, and hyperparameters across GA and LLM-driven workflows.

A key limitation of using GA for outer-loop optimization in meta-learning is search stagnation, where the algorithm stops finding improved loss functions or hyperparameters. This requires supplementing the GA search with LLM-driven prompt and primitive discovery to inject new capabilities.