fortify

To run ablation studies on completed ML research runs, use Fortify to automatically generate ablation variants from git diffs, execute them in isolated git worktrees, and rank component importance by comparing full vs ablated performance metrics.

Component ranking in ablation studies identifies which parts of a completed research run actually contribute to improvements by generating ablation variants, executing them in isolated git worktrees, and comparing performance metrics to rank each component's importance.

Fortify enforces isolation by creating dedicated git worktrees for each ablation variant while the main repository remains unchanged. This supports deterministic results, straightforward cleanup, and sandboxed experimentation without risking your primary working directory.

Yes, Fortify optionally generates reviewer Q&A material tailored to target venues like CVPR, NeurIPS, ICML, or workshops. This material is produced after ranking component importance from ablation results to help prepare responses for manuscript submissions.

Fortify identifies ablation candidates by analyzing git diffs and project diaries from a completed research run. It then creates isolated worktrees for each variant and runs metric and guard checks to evaluate each component's contribution.

Automated ablation studies with Fortify apply specifically to completed research runs and require git worktree support. The tool focuses on component ranking and reviewer Q&A generation, relying on existing best_metric sanity checks rather than creating new experimental configurations.