pyvene-interventions

Activation patching on PyTorch models is automated by defining an IntervenableConfig with RepresentationConfig entries to target specific layers and components. This declarative configuration specifies the intervention type, enabling systematic causal hypothesis testing on models like GPT-2 and LLaMA.

Causal interventions for neural network interpretability include VanillaIntervention, AdditionIntervention, SubtractionIntervention, CollectIntervention, RotatedSpaceIntervention, and LowRankRotatedSpaceIntervention. These configurable types enable diverse causal tracing and interchange intervention training workflows.

Causal tracing in transformer models is supported through declarative interventions that patch activations across targeted layers. By configuring representation parameters, researchers can systematically test causal hypotheses and trace specific model behaviors during generation.

Interchange intervention training for PyTorch models requires defining an IntervenableConfig with RepresentationConfig entries specifying the target layer and component. The framework supports end-to-end workflows for training, generation-time steering, and saving or loading the intervened models.

Saving and sharing intervened models on HuggingFace is fully supported. After applying causal interventions or interchange intervention training, the framework provides end-to-end workflows to save and load model states and share them via HuggingFace.

Declarative causal interventions on PyTorch models are set up by installing pyvene and configuring an IntervenableConfig with RepresentationConfig entries. This approach targets specific layers and components, providing a structured framework for activation patching and causal tracing.