sparse-autoencoder-training

To decompose dense neural network activations into interpretable features, train Sparse Autoencoders (SAEs) that map dense activations into sparse, monosemantic vectors, resolving polysemanticity and making model behavior easier to interpret.

Polysemanticity in neural networks occurs when individual neurons activate for multiple unrelated concepts. Sparse autoencoders resolve this by decomposing dense activations into sparse, monosemantic features, isolating distinct concepts for mechanistic interpretability.

Yes, integrating transformer-lens with sae-lens enables feature discovery and model steering. You can load pre-trained SAEs to identify top-activating features for a given prompt and manipulate them to influence model output toward or away from specific concepts.

To find top-activating features in gpt2-small, use this skill to load a pre-trained SAE via sae-lens and transformer-lens, pass your input prompt through the model, and extract the highest-activating sparse features from the encoded activations.

You do not need to train from scratch to perform feature attribution and ablation analysis. You can use pre-trained sparse autoencoders to analyze model behavior, perform feature attribution, and validate reconstruction quality across supported transformer architectures.

Sparse autoencoders for mechanistic interpretability require integration with transformer-lens and sae-lens to perform activation encoding and reconstruction. They depend on reconstruction quality validation and may not fully resolve all polysemantic neurons across complex transformer architectures.