autoresearch

Autonomous ML research automates hyperparameter tuning by iteratively modifying training code, executing short GPU experiments, and keeping or discarding changes based on the extracted validation metric. It runs autonomously overnight to optimize models.

Git-based experiment logging works by creating a dedicated branch, committing each training code modification, and recording validation metrics and peak VRAM in a results file. This keeps successful improvements while discarding failed runs.

Yes, you can run autonomous machine learning experiments with a fixed runtime budget by enforcing a strict time limit per training run. The system executes short GPU experiments and automatically evaluates whether to keep code changes based on the validation metric.

The best way to optimize val_bpb across multiple training runs is to use an autonomous experiment loop that modifies training code, executes short GPU experiments, and extracts the metric from logs to decide whether to keep or discard changes.

No, you do not need to modify prepare.py to run autonomous GPU training loops. You only edit train.py while treating prepare.py and evaluate_bpb as strictly read-only files during the automated experiment execution process.

Limitations of using autonomous experiments for model optimization include a strict fixed runtime budget per experiment and reliance on a single validation metric. The system requires tolerance for short training runs rather than continuous long executions.