skills.rest
tDalile
tDalileCOMMUNITY
Explore all skills by this author →
GitHub

autoresearch

Automate iterative ML experiments by modifying training code and extracting validation bits-per-byte.

Updated Aug 23, 2026
Search Tags:#gpu#training#gpt#experiment#ml#hyperparameter#autoresearch
One-click install
npx skills add https://github.com/tDalile/dotfiles --skill autoresearch-tdalile
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: autoresearch
Source: https://github.com/tDalile/dotfiles/tree/main/agents/skills/autoresearch
Command: npx skills add https://github.com/tDalile/dotfiles --skill autoresearch-tdalile

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates the repetitive and time-constrained process of proposing, applying, running, and evaluating short ML experiments to improve a small GPT model's validation bits-per-byte metric.

Core Features & Use Cases

  • Autonomous Experiment Loop: Reads and modifies train.py, runs training for a fixed five-minute budget, extracts val_bpb and memory usage, and logs results to a structured TSV.
  • Crash and Resource Management: Detects crashes, tails logs for quick fixes, enforces a 10-minute hard timeout, and constrains VRAM usage for low-memory GPUs (e.g., RTX 4050).
  • Versioned Tracking: Commits experimental changes to git, keeps commits that improve the metric, and rolls back unsuccessful attempts.
  • Use Case: Nightly hyperparameter and architecture search on a local workstation to squeeze performance out of constrained GPU hardware.

Quick Start

Start an autoresearch experiment by creating a new branch in the repository and instructing the agent to run a 5-minute training loop, evaluate val_bpb, and log results.

Frequently Asked Questions about autoresearch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate hyperparameter search for small GPT models on a local GPU?▼

You can automate hyperparameter optimization by running an autonomous experiment loop that modifies training code, executes five-minute runs, and logs validation bits-per-byte to a structured TSV file.

Can I run ML training experiments autonomously with under 6GB VRAM?▼

Yes, autonomous ML training experiments are designed for local GPUs with under 6GB VRAM, enforcing strict memory constraints and a ten-minute hard timeout to prevent resource exhaustion.

What is validation bits-per-byte optimization for GPT architectures?▼

Validation bits-per-byte optimization is an automated evaluation mechanism that extracts val_bpb metrics from short training runs to measure and improve small GPT model compression efficiency.

Does autoresearch require uv tooling and Python 3.10 to run experiments?▼

Yes, running automated ML experiments requires Python 3.10+, uv tooling, git access, and an NVIDIA GPU to properly execute the iterative training and code modification loops.

How does the agent handle crashes during automated ML training runs?▼

The system handles crashes during automated ML training by detecting failures, tailing logs for quick fixes, and rolling back unsuccessful git commits to maintain a stable experiment loop.

Related Skills

Explore More Software Engineering Skills→
JuanMS20JuanMS20

tensorrt-llm

Optimize LLM inference on NVIDIA GPUs with TensorRT-LLM.

Community
Advanced
RoentekRoentek
4

autoresearch

Iteratively modify train.py and run short GPU experiments to optimize val_bpb.

Official
Advanced
cfcostacfcosta

autoresearch-create

Set up and run autonomous experiment loops with logging and version control.

Community
Advanced
henrycashe26henrycashe26

ml-training-monitor

Diagnose machine learning training runs by interpreting loss curves and validation metrics.

Community
Advanced