transformer-lens-interpretability

Activation patching for mechanistic interpretability uses TransformerLens HookedTransformer APIs to load GPT-2 and run_with_cache, enabling causal tracing by swapping cached activations into specific hook points during inference.

TransformerLens is used for mechanistic interpretability research, exposing transformer activations and hook points to perform causal analysis, circuit discovery, and logit attribution across models like GPT-2.

Finding induction heads involves using TransformerLens to extract per-layer attention data from the activation cache, allowing you to identify and analyze specific circuit behaviors within the transformer model architecture.

Logit lens and attribution workflows can be applied to trace model outputs by accessing residual streams and attention patterns cached via the HookedTransformer API during the forward pass.

PyTorch is required alongside TransformerLens to inspect transformer internals, as HookedTransformer relies on PyTorch tensors to expose model activations and execute hook interventions for causal analysis.