skills.rest
handsomelong922
handsomelong922COMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations.

Updated Apr 3, 2026
Search Tags:#training#analyze#interpretability#feature-discovery#sae#sparse-autoencoder#neural-activations
One-click install
npx skills add https://github.com/handsomelong922/my-codex-skills --skill sparse-autoencoder-training-handsomelong922
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/handsomelong922/my-codex-skills/tree/main/skills/saelens
Command: npx skills add https://github.com/handsomelong922/my-codex-skills --skill sparse-autoencoder-training-handsomelong922

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens>=6.0.0, transformer-lens>=2.0.0, torch>=2.0.0, and includes references (resource) components.

What problem does it solve?

Provides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into sparse, interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.

Core Features & Use Cases

  • Training workflows: Load pre-trained SAEs, configure new SAEs, and train with architectures such as standard, gated, topk, or jumprelu.
  • Analysis workflows: Encode activations to sparse features, inspect top features per token, and evaluate reconstruction fidelity.
  • Steering & ablation: Use SAE features to steer model outputs or ablate features for causal analysis.
  • Integration & tutorials: Access references and tutorials in the references folder; leverage Neuronpedia and HuggingFace integrations.

Quick Start

Install SAE-Lens, configure a LanguageModelSAERunnerConfig, and run SAETrainingRunner to train and evaluate your first SAE.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a sparse autoencoder for interpretable feature discovery on transformer models?▼

To train a sparse autoencoder, you configure a LanguageModelSAERunnerConfig and execute SAETrainingRunner using the SAELens library to decompose neural activations into sparse, interpretable features across transformer models.

What sparse autoencoder architectures can I use to analyze neural activations?▼

You can analyze neural activations using standard, gated, topk, and jumprelu sparse autoencoder architectures, configuring each via SAELens to evaluate reconstruction fidelity and inspect top features per token.

Can I use pre-trained sparse autoencoders with HuggingFace and Neuronpedia for feature analysis?▼

Yes, you can load pre-trained sparse autoencoders and perform feature analysis by integrating SAELens with HuggingFace and Neuronpedia to inspect monosemantic representations and study superposition in language models.

Do I need PyTorch and TransformerLens to steer model outputs using sparse autoencoder features?▼

Yes, steering model outputs and ablating features for causal analysis requires PyTorch, TransformerLens, and SAELens to encode activations to sparse features and manipulate them during inference.

What is the best way to decompose neural activations into monosemantic representations?▼

The best way to decompose neural activations into monosemantic representations is training sparse autoencoders with SAELens, which supports multiple architectures to discover interpretable features and evaluate reconstruction fidelity.

Why use sparse autoencoders instead of directly interpreting transformer model neurons?▼

Sparse autoencoders resolve superposition by decomposing dense neural activations into sparse, interpretable features, allowing you to isolate monosemantic representations that direct neuron analysis cannot easily disentangle.

Related Skills

Explore More Data & Analytics Skills→
cloudlinesscloudliness

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens on transformer activations.

Community
Advanced
carterwayneskhizeinecarterwayneskhizeine
19

sparse-autoencoder-training

Train and analyze sparse autoencoders with SAELens on TransformerLens activations.

Community
Advanced
twjarviszyra-webtwjarviszyra-web
3

sparse-autoencoder-training

Train and analyze sparse autoencoders for neural network activations.

Community
Advanced
OpenLAIROpenLAIR
1.0k

transformer-lens-interpretability

Analyze and steer transformer activations with TransformerLens for mechanistic interpretability.

Official
Advanced