nnsight-remote-interpretability

Remote interpretability enables deferred tracing and remote inspection of neural network internals. By using NDIF remote execution via a LanguageModel wrapper, you can capture and analyze activations on 70B+ parameter models without exhausting local hardware resources.

Activation patching involves intervening in neural network internals by modifying or sharing captured activations between different forward passes. Multi-prompt tracing allows you to save proxy object activations from one input and apply them during another, enabling cross-prompt interventions and generation-time behavior analysis.

Yes, you can trace large neural models remotely using NDIF. The unified tracing API supports deferred execution to scale from small local experiments to 70B+ parameter models, allowing you to collect and analyze gradients and activations without local hardware limitations.

Remote interpretability works with PyTorch-based architectures by utilizing a LanguageModel wrapper and trace context. You instantiate the model, begin a trace to capture layer activations, and use proxy objects to access and save internal data for analysis.

The best way to save and analyze gradients from a 70B model is to use remote execution with NDIF. Deferred tracing allows you to capture gradients via proxy objects during the trace context, saving the results remotely for later analysis without local memory constraints.

Deferred tracing should be used instead of local execution when interpreting large neural network internals that exceed local hardware capacity. It is essential for scaling activation analysis and intervention to 70B+ models via NDIF, preventing out-of-memory errors during remote inspection.