transformer-lens-interpretability

Activation patching runs the model on a clean prompt with run_with_cache, then patches those activations into a corrupted run via run_with_hooks. Measure the effect with a metric like logit difference and iterate over layers and positions to build a heatmap of causal hotspots.

TransformerLens supports over 50 models including GPT-2, LLaMA, LLaMA-2, Mistral, Mixtral, Pythia, GPT-Neo, GPT-J, OPT, Gemma, Phi, and Qwen. Gated models like LLaMA require setting an HF_TOKEN environment variable before loading.

TransformerLens is the standard for GPT-style transformer circuit analysis with HookPoints on every activation. Use nnsight for remote execution on massive models or non-transformer architectures, and pyvene for higher-level causal intervention abstractions.

Hooks persist because old hooks remain active after run_with_hooks calls during debugging. Call model.reset_hooks() before each new hooked run to clear previously registered hooks and avoid stale modifications.

Use the names_filter parameter in run_with_cache to cache only needed activations, such as resid_post or pattern. You can also pass device="cpu" to store the cache in CPU memory instead of GPU memory.

Yes, TransformerLens integrates with SAELens for Sparse Autoencoder analysis. Load an SAE with SAE.from_pretrained, cache activations from a hook point like blocks.8.hook_resid_pre, and encode them with sae.encode.