sparse-autoencoder-training

To train sparse autoencoders on transformer activations, load a pretrained model and configure an SAE with your preferred architecture. The workflow handles activation loading, sparsity control, and integration with TransformerLens via SAELens APIs to produce interpretable features.

Standard, gated, and top-k are SAE architectures you can compare to assess the trade-off between sparsity and reconstruction quality. Evaluating these different architectures helps identify the most interpretable feature representations for mechanistic interpretability research.

Yes, you can use TransformerLens and SAELens to extract interpretable features. The Skill integrates directly with SAELens APIs to load model activations, train sparse autoencoders, and evaluate the resulting sparse, monosemantic feature sets.

You evaluate sparsity vs reconstruction quality by comparing different SAE architectures during the training workflow. The process calculates reconstruction metrics to ensure the sparse autoencoder accurately represents the original dense neural activations.

Sparse autoencoders decompose dense neural activations into sparse, interpretable features to address polysemanticity. This decomposition reveals monosemantic representations, enabling researchers to understand model behavior, debug internal mechanisms, and conduct safety studies.

Sparse autoencoders for model debugging require balancing sparsity against reconstruction quality. If reconstruction is poor, the extracted interpretable features may not accurately reflect the underlying transformer activations, limiting the reliability of steering experiments and safety studies.