sparse-autoencoder-training

To train a Sparse Autoencoder, you use SAELens to specify model hooks, expansion factors, and sparsity penalties. This decomposes neural activations into interpretable, monosemantic features for your transformer model.

Analyzing superposition involves training Sparse Autoencoders with SAELens to dissect neural activations. This process reveals interpretable features and monosemantic representations hidden within the language model's superposition.

Feature attribution and steering are performed by loading a pre-trained SAE with SAELens onto a model like GPT-2 small. You encode activations and identify top activating features to steer model behavior across prompts.

Yes, you can compare features across different prompts and integrate with Neuronpedia. SAELens supports cross-prompt comparisons and Neuronpedia integration to evaluate and visualize discovered interpretable features.

Sparse Autoencoder training requires specifying training tokens, sparsity penalties, and expansion factors. You also define evaluation metrics to properly assess the quality of interpretable features extracted from the model hooks.

Ablation experiments are run by loading a pre-trained or custom SAE with SAELens and targeting specific neural activations. This allows you to measure the causal impact of identified interpretable features on model outputs.