autoresearch

Automate autonomous ML experiments by using a closed-loop workflow that edits train.py, runs 300-second experiments, and logs results. This enables continuous overnight hyperparameter exploration on single-GPU Linux setups.

Ensure reproducible ML research by using git ratcheting to lock in model improvements and a results.tsv log to record val_bpb and memory usage. A program.md directive and locked dependencies maintain a tamper-evident history.

Yes, you can run architecture ablations and baseline evaluation without manual edits. The agent autonomously edits train.py, executes fixed-time experiments, and performs git-based ratcheting to monotonically improve models.

Yes, you need to lock dependencies in pyproject.toml before starting closed-loop ML experimentation. The immutable evaluation harness and reproducible setup rely exclusively on these locked dependencies.

The 300-second budget limits each experiment iteration to short architecture and hyperparameter tests. This fixed-time constraint is designed for single-GPU Linux setups and may not suit long-running training regimes.

An immutable evaluation harness is required to ensure fair, reproducible comparisons across all iterations. It prevents evaluation logic changes from confounding the val_bpb results recorded in your results.tsv log.