sparse-autoencoder-training

To train a sparse autoencoder on GPT-2 activations, use SAELens with configurable architecture and sparsity settings. This process decomposes transformer activations into interpretable features for mechanistic interpretability analysis.

Mechanistic interpretability analyzes model internals by decomposing neural activations into interpretable features. Superposition allows models to represent more features than dimensions, which sparse autoencoders help untangle by identifying these overlapping monosemantic features.

Yes, analyzing sparse autoencoders requires Python 3.10+ along with the SAELens and transformer-lens libraries. These dependencies provide the necessary environment to load pre-trained SAEs and perform feature ablation end-to-end.

Yes, you can load and apply pre-trained SAEs to model activations for feature steering and ablation. This allows cross-prompt comparisons and interpretability analysis across GPT-2 or GEMMA-like models without training a new sparse autoencoder.

The best way to discover monosemantic features is training sparse autoencoders with SAELens to decompose activations. This isolates interpretable features from superposition, enabling feature-based steering and ablation within transformer models.

Avoid using sparse autoencoders when lacking Python 3.10+ or SAELens and transformer-lens dependencies. Without these environments, loading pre-trained SAEs or training custom SAEs on transformer activations cannot operate end-to-end.