sparse-autoencoder-training

To decompose neural network activations into interpretable features, you train a Sparse Autoencoder to resolve polysemantic activations into sparse, monosemantic representations. This process provides a clear window into the underlying concepts learned by language models.

Feature superposition in language models occurs when multiple concepts overlap within a single activation space. You analyze this superposition by training Sparse Autoencoders to untangle these dense representations into distinct, monosemantic features for interpretability research.

Yes, you can use transformer-lens and sae-lens to analyze language model internal representations. This Skill relies on these specific libraries alongside PyTorch to extract and train Sparse Autoencoders directly on model activations for interpretability.

The best way to train a Sparse Autoencoder for interpretability is to extract internal activations from a language model and apply SAE training scripts to map them. This decomposes complex activations into sparse features that represent distinct concepts.

Yes, you need PyTorch version 2.0.0 or higher to train Sparse Autoencoders for neural network interpretability. The training process depends on PyTorch alongside the sae-lens and transformer-lens libraries to process activations effectively.