skills.rest
JuanMS20
JuanMS20COMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and evaluate Sparse Autoencoders on transformer activations using sae-lens and transformer-lens.

Updated May 3, 2026
Search Tags:#training#interpretability#sae#transformer-lens#sparse-autoencoder#model-analysis
One-click install
npx skills add https://github.com/JuanMS20/solviora-agent --skill sparse-autoencoder-training-juanms20
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/JuanMS20/solviora-agent/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/JuanMS20/solviora-agent --skill sparse-autoencoder-training-juanms20

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Provides a structured approach to training Sparse Autoencoders (SAEs) to decompose transformer activations into sparse, interpretable features, enabling deeper mechanistic insights into model behavior.

Core Features & Use Cases

  • Guided workflows for selecting SAE architecture, expansion factors, and training hyperparameters.
  • Tools to evaluate sparsity, reconstruction accuracy, and feature interpretability across prompts.
  • Integration with TransformerLens and neuronpedia-style analysis for feature attribution and exploration.

Quick Start

Follow the tutorials to begin training an SAE on a GPT-2 activation stream and evaluate sparsity and reconstruction.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a sparse autoencoder on transformer activations?▼

To train a sparse autoencoder on transformer activations, configure the SAE architecture and expansion factors, load a model using TransformerLens, and evaluate sparsity and reconstruction quality across prompts.

Why use sparse autoencoders for model interpretability?▼

Sparse autoencoders decompose transformer activations into sparse, interpretable features, enabling deeper mechanistic insights into model behavior by isolating specific activation patterns for analysis.

Does this sparse autoencoder training workflow support multiple SAE architectures?▼

Yes, this sparse autoencoder training workflow supports multiple SAE architectures, allowing you to select different configurations and expansion factors while including safety checks and logging during training.

How do I evaluate SAE reconstruction accuracy and feature interpretability?▼

Evaluate SAE reconstruction accuracy and feature interpretability by analyzing sparsity metrics and reconstruction quality across prompts, utilizing neuronpedia-style analysis for feature attribution and exploration.

Do I need sae-lens and transformer-lens to train SAEs for model analysis?▼

Yes, you need the sae-lens and transformer-lens libraries to train SAEs for model analysis, as they provide the required interfaces for loading models and guiding the interpretability workflows.

What is the best way to start training an SAE on GPT-2 activation streams?▼

The best way to start training an SAE on GPT-2 activation streams is to follow the guided tutorials to configure hyperparameters, train the model, and evaluate sparsity and reconstruction.

Related Skills

Explore More Data & Analytics Skills→
letta-ailetta-ai
134

gpt2-codegolf

Implement minimal GPT-2 inference with checkpoint parsing and BPE tokenization.

Official
Advanced
carterwayneskhizeinecarterwayneskhizeine
19

sparse-autoencoder-training

Train and analyze sparse autoencoders with SAELens on TransformerLens activations.

Community
Advanced
ruiyangruiyiruiyangruiyi

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations into sparse features.

Community
Advanced
dawsonblockdawsonblock

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced