skills.rest
AissenLiu
AissenLiuCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens on transformer activations.

Updated Apr 27, 2026
Search Tags:#transformers#interpretability#feature-discovery#sae#sae-lens#mechanistic
One-click install
npx skills add https://github.com/AissenLiu/EasyHermes --skill sparse-autoencoder-training-aissenliu
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/AissenLiu/EasyHermes/tree/main/hermes-agent/optional-skills/mlops/saelens
Command: npx skills add https://github.com/AissenLiu/EasyHermes --skill sparse-autoencoder-training-aissenliu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens>=6.0.0, transformer-lens>=2.0.0, torch>=2.0.0, and includes references (resource) components.

What problem does it solve?

Trains and analyzes Sparse Autoencoders (SAEs) with SAELens to decompose neural activations into sparse, interpretable features.

Core Features & Use Cases

  • Load pre-trained SAEs for analysis and experimentation.
  • Train custom SAEs on transformer activations to study monosemantic features and superposition.
  • Perform feature analysis, steering, and ablation to understand causal contributions in model outputs.

Quick Start

Load a model, attach an SAE, and begin analyzing sparse features in activations.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train sparse autoencoders to decompose transformer activations into interpretable features?▼

You can train custom SAEs on transformer activations using SAELens to study monosemantic features and superposition. This allows you to decompose neural activations into sparse, interpretable features for mechanistic interpretability research.

What is the best way to analyze monosemantic features in a transformer model?▼

Analyzing monosemantic features involves training custom SAEs on transformer activations with SAELens. You can then perform feature analysis, steering, and ablation to understand causal contributions in model outputs.

Can I load pre-trained SAEs for feature analysis and experimentation?▼

Yes, you can load pre-trained SAEs using SAELens for analysis and experimentation. This allows you to immediately begin analyzing sparse features and performing ablation studies on transformer activations.

Do I need transformer-lens and PyTorch to perform feature discovery with sparse autoencoders?▼

Yes, performing feature discovery with sparse autoencoders requires transformer-lens, PyTorch, and sae-lens. You also need Python 3.10 or higher to execute the documented training and analysis workflows safely.

How does sparse autoencoder training help with causal analysis in model outputs?▼

Sparse autoencoder training helps with causal analysis by decomposing activations into interpretable features. You can then perform feature steering and ablation to directly understand and test causal contributions in model outputs.

Related Skills

Explore More Education & Research Skills→
letta-ailetta-ai
134

gpt2-codegolf

Implement minimal GPT-2 inference with checkpoint parsing and BPE tokenization.

Official
Advanced
SignmanalSignmanal
1

sparse-autoencoder-training

Decompose polysemantic transformer activations into sparse interpretable features with SAELens.

Community
Advanced
cloudlinesscloudliness

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens on transformer activations.

Community
Advanced
cxnaivecxnaive

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose dense neural activations into sparse features.

Community
Advanced