nnsight-remote-interpretability

Mechanistic interpretability on massive foundation models is possible via remote execution through NDIF, allowing you to analyze models up to 405B parameters without local GPUs. This Skill enables activation analysis, patching, and gradient-based studies by sending execution requests to remote hardware.

Yes, you can access and modify PyTorch model internals including activations, gradients, and attention patterns. This Skill provides a unified interface to manipulate the internal workings of any PyTorch architecture, supporting deep model understanding and intervention.

This approach works with universal PyTorch support, meaning it handles any PyTorch architecture including Transformers, Mamba, and custom models. It leverages the nnsight library to trace and introspect model internals regardless of the specific neural network design.

To start analyzing neural network internals, you need the nnsight and torch libraries installed in your Python environment. These dependencies provide the foundational PyTorch model introspection capabilities required to trace models and extract hidden states.

To trace a model like gpt2 and save hidden states, use the nnsight interface to pass your prompt and specify the target layer. This Skill intercepts the forward pass, allowing you to extract and save internal activations from specific layers for analysis.

Remote execution via NDIF is limited to supported models and requires network connectivity, sending model execution requests to remote hardware. While it eliminates local GPU requirements for massive models, execution speed may depend on remote resource availability and queue times.