Verified:GitHub

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer language model activations.

Updated Apr 10, 2026

Search Tags:#python #feature-analysis #mechanistic-interpretability #transformer-lens #sparse-autoencoders #model-steering

Ready to run

npx skills add https://github.com/KarlinskyS/hermesSkills --skill sparse-autoencoder-training-karlinskys

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/KarlinskyS/hermesSkills/tree/main/mlops/evaluation/saelens
Command: npx skills add https://github.com/KarlinskyS/hermesSkills --skill sparse-autoencoder-training-karlinskys

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Modern transformer models exhibit polysemantic neurons and superposition, making it hard to interpret learned concepts. This Skill provides tools to train and analyze Sparse Autoencoders (SAEs) that decompose dense activations into sparse, interpretable features, enabling researchers to discover and steer model behavior.

Core Features & Use Cases

Load and explore pre‑trained SAEs: Quickly retrieve and inspect feature activations for any transformer layer.
Train custom SAEs: Configure architecture, sparsity, and training hyper‑parameters to fit specific datasets or layers.
Feature analysis & steering: Identify top‑activating features, attribute them to model outputs, and perform causal interventions.
Integration with Neuronpedia: Browse discovered features in an online catalog for cross‑model comparison.

Quick Start

Use the SAELens skill to load a GPT‑2 model and display the top five SAE features for the token 'Paris'.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract interpretable features from transformer model activations?▼

To extract interpretable features from transformer activations, you train sparse autoencoders that decompose dense activations into sparse, monosemantic features, enabling feature discovery and superposition analysis on models like GPT-2.

How does sparse autoencoder training help with mechanistic interpretability?▼

Sparse autoencoder training aids mechanistic interpretability by resolving polysemantic neuron superposition, decomposing dense model activations into interpretable features for causal interventions and model steering.

Can I use SAELens and TransformerLens to analyze pre-trained sparse autoencoders?▼

Yes, you can use SAELens with TransformerLens to load and inspect pre-trained sparse autoencoders, quickly retrieving feature activations for any transformer layer without training from scratch.

Do I need a GPU to train sparse autoencoders on language models?▼

Yes, a GPU-enabled environment is required for efficient computation when you train custom sparse autoencoders on language model activations, along with Python 3.10+ and the sae-lens library.

What is the best way to steer model behavior using discovered features?▼

The best way to steer model behavior is identifying top-activating sparse autoencoder features, attributing them to model outputs, and performing causal interventions to directly manipulate specific concepts.

How do I browse discovered SAE features for cross-model comparison?▼

You browse discovered SAE features for cross-model comparison by integrating with Neuronpedia, an online catalog where discovered sparse autoencoder features are stored and visualized.