nnsight-remote-interpretability

Activation patching on large language models requires tracing PyTorch model internals, saving intermediate activations via proxy, and intervening across layers or prompts to test causal hypotheses. You can execute these interventions remotely on 70B+ models using NDIF.

Mechanistic analysis of neural network internals involves tracing and reading intermediate activations, attention patterns, and logits directly from model modules to understand behavior. It enables activation analysis and causal investigation of layer and position effects.

You can run interpretability experiments on 70B+ models remotely by enabling remote execution within the trace context. NDIF infrastructure supports multi-prompt workflows and generation-time interventions without local compute constraints.

You need PyTorch and nnsight with a LanguageModel wrapper to trace and save model activations. The trace context and proxy .save() method are used to capture intermediate outputs for interpretability experiments and causal analysis.

Yes, cross-prompt activation sharing and generation-time interventions are supported. You can patch activations across different prompts and manipulate internal states during generation to investigate causal effects on model outputs.

Remote execution for model interpretability is limited to activation analysis, patching, and generation-time interventions. Gradient-based analysis is only available for local execution, not remote NDIF runs on massive models.