skills.rest
overviewlabs
overviewlabsCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and evaluate Sparse Autoencoders on transformer activations with SAELens and TransformerLens.

Updated Apr 10, 2026
Search Tags:#training#transformer#mechanistic-interpretability#sae#sae-lens#sparse-autoencoders#activation-analysis
One-click install
npx skills add https://github.com/overviewlabs/WHOX --skill sparse-autoencoder-training-overviewlabs
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/overviewlabs/WHOX/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/overviewlabs/WHOX --skill sparse-autoencoder-training-overviewlabs

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This training skill provides a guided workflow to train and evaluate Sparse Autoencoders (SAEs) on Transformer activations, enabling decomposition of dense representations into sparse, interpretable features for mechanistic interpretability research.

Core Features & Use Cases

  • Configurable SAE architectures (Standard, Gated, TopK, JumpReLU) for modeling neural activations.
  • End-to-end training, evaluation, and analysis workflows with SAELens, TransformerLens, and Neuronpedia.
  • Use Cases: discover interpretable features, study superposition, and validate feature-based interventions in language models.

Quick Start

Run the training workflow to train an SAE on GPT-style activations using a chosen model and hook.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train sparse autoencoders on transformer activations for mechanistic interpretability?▼

You can use Standard, Gated, TopK, or JumpReLU architectures for sparse autoencoder training. These configurable SAE architectures model neural activations differently to decompose dense representations into sparse, interpretable features.

Can I inspect SAE features in Neuronpedia after training on transformer activations?▼

Sparse autoencoders are used to study superposition by decomposing dense transformer representations into sparse, interpretable features. This allows researchers to discover interpretable features and validate feature-based interventions in language models.

What is the best way to configure sparsity control when training SAEs?▼

The best way to configure sparsity control during SAE training is through the provided workflow tooling, which satisfies configuration requirements including architecture selection and sparsity control to effectively isolate interpretable features.

Does this sparse autoencoder training workflow support feature steering in language models?▼

Yes, this sparse autoencoder training workflow supports feature steering in language models. After isolating sparse features, you can validate feature-based interventions to study and manipulate model behavior mechanistically.

Related Skills

Explore More Education & Research Skills→
jesseotremblayjesseotremblay
1

analyzing-business-models

Analyze business models using the Business Model Canvas framework.

Community
Basic
carterwayneskhizeinecarterwayneskhizeine
19

sparse-autoencoder-training

Train and analyze sparse autoencoders with SAELens on TransformerLens activations.

Community
Advanced
zerwizzerwiz

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer activations with SAELens.

Community
Advanced
icyzhicyzh
1

sparse-autoencoder-training

Train sparse autoencoders to decompose transformer activations into interpretable features.

Community
Advanced