skills.rest
dawsonblock
dawsonblockCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Updated Apr 26, 2026
Search Tags:#interpretability#mechanistic-interpretability#transformer-lens#sae-lens#sparse-autoencoders#activation-analysis
One-click install
npx skills add https://github.com/dawsonblock/HERMY --skill sparse-autoencoder-training-dawsonblock
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/dawsonblock/HERMY/tree/main/hermes-agent-2026.4.23/optional-skills/mlops/saelens
Command: npx skills add https://github.com/dawsonblock/HERMY --skill sparse-autoencoder-training-dawsonblock

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Train and analyze Sparse Autoencoders (SAEs) using SAELens to decompose dense transformer activations into sparse, interpretable features for mechanistic interpretability research.

Core Features & Use Cases

  • Load pre-trained SAEs and encode model activations to sparse features.
  • Decode features to reconstruct activations and measure reconstruction quality.
  • Perform feature attribution, steering, and ablation to study model behavior and safety.

Quick Start

Train and analyze Sparse Autoencoders on a transformer model using SAELens to extract interpretable features.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train sparse autoencoders on transformer activations for mechanistic interpretability?▼

Load pre-trained sparse autoencoders with SAELens to encode transformer model activations into sparse features. You can then decode these features to reconstruct activations and measure reconstruction quality for mechanistic interpretability analysis.

Can I perform feature attribution and steering on transformer models with sparse autoencoders?▼

Perform feature attribution, steering, and ablation using SAELens to study and manipulate transformer model behavior. This identifies how specific sparse features influence model safety and outputs during mechanistic interpretability research.

What Python environment do I need to train sparse autoencoders with SAELens?▼

Training sparse autoencoders with SAELens requires Python 3.10 or higher, along with SAELens, TransformerLens, and PyTorch. A GPU-enabled environment is recommended for training on GPT-2–sized transformer models.

How does SAELens decompose dense transformer activations into interpretable features?▼

SAELens decomposes dense transformer activations into sparse, interpretable features by encoding the activations through a sparse autoencoder. This process isolates individual features that represent specific concepts within the model.

Are sparse autoencoders suitable for analyzing GPT-2-sized transformer models?▼

Yes, sparse autoencoders are suitable for analyzing GPT-2-sized transformer models when using a GPU-enabled environment. SAELens supports loading pre-trained SAEs and training new ones for models of this scale.

Related Skills

Explore More Education & Research Skills→
SethyPagnaSethyPagna

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer language model activations.

Community
Advanced
SignmanalSignmanal
1

sparse-autoencoder-training

Decompose polysemantic transformer activations into sparse interpretable features with SAELens.

Community
Advanced
DaddyElonMusk69DaddyElonMusk69

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced
jongwonyjongwony
4

claude-code-internals

Locate Claude Code installations and inspect minified source behavior.

Community
Advanced