skills.rest
adm-humanerd
adm-humanerdCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose dense activations into sparse features.

Updated Aug 27, 2026
Search Tags:#interpretability#sae#saelens#sparse-autoencoder#neural-activations#mechanistic
One-click install
npx skills add https://github.com/adm-humanerd/drewgent --skill sparse-autoencoder-training-adm-humanerd
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/adm-humanerd/drewgent/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/adm-humanerd/drewgent --skill sparse-autoencoder-training-adm-humanerd

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

SAE training and analysis to decompose dense neural activations into sparse, interpretable features using SAELens, enabling researchers to study how models represent information.

Core Features & Use Cases

  • Load pre-trained SAEs and encode activations from models using TransformerLens
  • Train custom SAEs with configurable architectures, sparsity controls, and evaluation metrics
  • Analyze features, perform steering/ablation experiments, and visualize feature relevance for predictions
  • Refer to detailed tutorials and reference materials in the references folder

Quick Start

Load a pre-trained SAE, encode a sample activation, and inspect the resulting sparse features.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a sparse autoencoder to decompose neural activations into interpretable features?▼

To train a sparse autoencoder, use SAELens to configure architectures and sparsity controls, then run training workflows on dense neural activations extracted via TransformerLens to produce sparse, interpretable features.

How do I load a pre-trained sparse autoencoder and encode activations from a language model?▼

Load pre-trained sparse autoencoders using SAELens and encode model activations by passing dense activations from TransformerLens through the SAE to inspect the resulting sparse feature representations.

Do I need Python 3.10 and specific libraries to run sparse autoencoder training workflows?▼

Yes, sparse autoencoder training requires Python 3.10 or higher, along with the transformer-lens and sae-lens libraries, to execute the training and analysis workflows for mechanistic interpretability research.

Can I perform steering or ablation experiments on sparse autoencoder features?▼

Yes, you can perform steering and ablation experiments on sparse autoencoder features to analyze their relevance for predictions and visualize how specific interpretable features influence model behavior.

What is the best way to analyze monosemantic representations and superposition in language models?▼

Analyzing monosemantic representations and superposition involves training sparse autoencoders with SAELens to decompose dense activations, then evaluating the resulting sparse features for interpretability and feature relevance.

Related Skills

Explore More Education & Research Skills→
letta-ailetta-ai
134

gpt2-codegolf

Implement minimal GPT-2 inference with checkpoint parsing and BPE tokenization.

Official
Advanced
cxnaivecxnaive

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose dense neural activations into sparse features.

Community
Advanced
AlexiosBluffMaraAlexiosBluffMara
2

sparse-autoencoder-training

Train and analyze sparse autoencoders with SAELens for transformer interpretability.

Community
Advanced
gqf2008gqf2008

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations.

Community
Intermediate