sparse-autoencoder-training

To decompose neural network activations into interpretable features, you train Sparse Autoencoders to resolve polysemanticity, breaking dense activations down into sparse, monosemantic features that correspond to human-interpretable concepts.

Polysemanticity occurs when individual neurons activate for multiple unrelated concepts, obscuring model behavior. You resolve it by training Sparse Autoencoders to map these dense activations into sparse, monosemantic, human-interpretable features.

Causal steering and ablation are performed by using Sparse Autoencoders to identify specific interpretable features in transformer activations, then intervening by steering or ablating those features to observe their impact on model output.

Yes, you need transformer-lens and torch to analyze superposition and feature geometry. These dependencies, alongside sae-lens, provide the required computational framework for high-dimensional activation decomposition and reconstruction.

You can load a pre-trained Sparse Autoencoder for GPT-2 using the sae-lens library to identify top-activating features for a given input prompt, enabling immediate feature discovery and model analysis without training from scratch.

While sparse autoencoders effectively decompose activations into interpretable features, they require high-dimensional activation reconstruction and depend on heavy libraries like torch and transformer-lens, making them resource-intensive for large-scale deployment.