sparse-autoencoder-training

You can decompose neural network activations into interpretable features by training sparse autoencoders. This process resolves polysemanticity by breaking dense model activations down into sparse, monosemantic features that correspond to human-interpretable concepts.

To identify features responsible for bias in language models, use sparse autoencoders for feature discovery. This allows researchers to isolate specific sparse, monosemantic features representing safety-relevant behaviors, enabling targeted steering or ablation experiments.

You can analyze superposition and feature geometry by using sparse autoencoders to decompose dense activations. This reveals how models represent overlapping information, helping you study the underlying geometry of feature representations within transformer models.

Yes, you can use sae-lens and transformer-lens to load a pre-trained sparse autoencoder for gpt2-small. This allows you to quickly identify top-activating features for a given input prompt without training a new model from scratch.

Individual neurons represent multiple overlapping concepts due to polysemanticity, making model behavior difficult to interpret. Sparse autoencoders solve this by decomposing these dense, polysemantic activations into sparse, monosemantic, and human-interpretable features.

Yes, you need torch along with sae-lens and transformer-lens to perform high-dimensional activation decomposition and reconstruction. These dependencies provide the necessary computational backend for training and analyzing sparse autoencoders.