pytorch-aot

PyTorch AOT Autograd works by tracing the forward pass to create a joint graph, partitioning it into forward and backward passes, and applying post-grad optimizations. This pipeline ensures autograd correctness while enabling ahead-of-time graph transformations.

Adding custom batching rules for vmap in functorch requires defining how your specific operator handles batched dimensions. You must implement the rule within the functorch batching framework to ensure correct gradient transforms and Core ATen IR transformations.

Joint graph construction in PyTorch AOT Autograd fuses forward and backward computation to capture full gradient transforms. Partitioning then splits this joint graph to save activations and apply post-grad FX passes for optimized runtime compilation.

Yes, PyTorch AOT Autograd supports custom post-grad optimization passes. You can extend the pipeline by adding FX passes that run after the joint graph is partitioned, enabling targeted optimizations for training and inference paths.

A custom vmap operator fails autograd correctness checks when mutation tracking or GraphSignature consistency is broken. You must verify that your functorch batching rules and decompositions align with Core ATen IR expectations across training paths.

You need Core ATen IR transformations for PyTorch functional modules when implementing decompositions or extending functionalization. This ensures the operations remain compatible with the AOT Autograd compilation pipeline and vmap gradient transforms.