sparse-autoencoder-training

To decompose neural network activations into interpretable features, train a sparse autoencoder using SAELens to isolate monosemantic representations. This process resolves polysemanticity by breaking dense activations into sparse, interpretable features for analysis.

Superposition in mechanistic interpretability occurs when a neural network represents more features than it has dimensions by compressing dense activations. You analyze it by training a sparse autoencoder to decompose model activations and study the resulting feature geometry.

Train a custom sparse autoencoder on GPT-2 small model activations by using SAELens alongside transformer-lens and torch. This Skill provides the necessary scripts and guidance to hook into model activations and initiate SAE training for feature discovery.

Yes, SAELens works with Gemma models for feature discovery and feature-based steering. It facilitates identifying specific interpretable features within Gemma activations, allowing you to analyze learned representations and guide model behavior.

Yes, you can use transformer-lens and torch to identify monosemantic representations by training a sparse autoencoder on model activations. This combination enables you to extract interpretable features that activate for specific concepts like legal language or HTTP requests.

Neural networks exhibit polysemanticity when individual neurons activate for multiple unrelated concepts due to superposition. You need a sparse autoencoder when analyzing language model behavior to untangle these dense activations into distinct, interpretable features.