What problem does it solve? Neural network neurons are polysemantic, activating for many unrelated concepts due to superposition, which makes models hard to interpret. This Skill guides you through using SAELens to decompose dense model activations into sparse, monosemantic features that correspond to interpretable concepts. ## Core Features & Use Cases - Pre-trained SAE Analysis: Load SAEs from releases like gpt2-small-res-jb, encode activations into sparse features, and identify top-activating features per token. - Custom SAE Training: Configure and train Standard, Gated, TopK, or JumpReLU SAEs with the v6 nested config API, monitoring L0, CE loss recovery, and dead feature ratios. - Feature Steering and Attribution: Add decoder feature directions to the residual stream for steering, compute per-feature logit contributions, and ablate features to test causal importance. - Use Case: You want to understand what GPT-2 Small has learned at layer 8. Load the matching pre-trained SAE, encode activations from your prompts, and inspect which interpretable features fire, then steer generation by amplifying a chosen feature direction. ## Quick Start Ask the agent to load the gpt2-small-res-jb sparse autoencoder with SAELens and show the top activating features for each token in a sample prompt.
