What problem does it solve?
This Skill helps users verify the correctness of their autoimprove.yaml configuration, diagnose failures in gates and benchmark metrics, and troubleshoot extraction issues, preventing costly mistakes during automated optimization processes.
Core Features & Use Cases
- Config Validation: Checks the structural integrity and required fields of autoimprove.yaml.
- Gate & Benchmark Dry Runs: Executes commands to ensure gates complete successfully and benchmarks produce expected outputs.
- Metric Extraction Debugging: Tests regex or jq extraction patterns against live benchmark outputs to identify issues with data parsing.
- Use Case: A developer running autoimprove encounters unexplained missing metrics; diagnose runs a thorough check of config, gate commands, and metric patterns to identify mismatches or errors.
Quick Start
Use the diagnose skill to validate your autoimprove.yaml configuration and test your gate commands and metric extraction patterns before starting autoimprove experiments.