skills.rest
math-inc
math-incOFFICIAL
Explore all skills by this author →
Website𝕏@mathematics_incGitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders on transformer activations with SAELens.

1.2k|116|Updated Mar 19, 2026
Search Tags:#mechanistic-interpretability#sae#transformer-lens#sparse-autoencoder#interpretable-activations#neuronpedia
One-click install
npx skills add https://github.com/math-inc/OpenGauss --skill sparse-autoencoder-training-math-inc
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/math-inc/OpenGauss/tree/main/skills/mlops/evaluation/saelens
Command: npx skills add https://github.com/math-inc/OpenGauss --skill sparse-autoencoder-training-math-inc

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens>=6.0.0, transformer-lens>=2.0.0, torch>=2.0.0, and includes references (resource) components.

What problem does it solve?

Provides a comprehensive workflow for training and using Sparse Autoencoders (SAEs) to decompose neural activations into sparse, interpretable features, enabling mechanistic interpretability research and feature-based analysis.

Core Features & Use Cases

  • Load pre-trained SAEs and inspect activations
  • Train custom SAEs with configurable architectures and sparsity
  • Analyze, steer, and ablate SAE features to understand model behavior
  • Integrate with TransformerLens and Neuronpedia for end-to-end analysis

Quick Start

Load a pre-trained SAE and encode activations from a model to begin discovering sparse, interpretable features.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a sparse autoencoder for transformer interpretability?▼

Train sparse autoencoders (SAEs) using the SAELens library to decompose neural activations into interpretable features for mechanistic interpretability research. You can configure custom architectures and sparsity levels to analyze model behavior.

What are sparse autoencoders used for in mechanistic interpretability?▼

Sparse autoencoders (SAEs) encode neural activations into sparse, interpretable feature directions, enabling mechanistic interpretability research, model debugging, and feature-based analysis across transformer models.

Can I load pre-trained SAEs and inspect activations with TransformerLens?▼

Yes, you can load pre-trained SAEs and inspect activations through the SAELens and TransformerLens integration. This allows you to immediately begin discovering sparse, interpretable features without training from scratch.

How do I analyze and steer SAE features to understand model behavior?▼

Analyze, steer, and ablate SAE features to understand model behavior using the integrated SAELens workflow. This feature-based analysis helps reveal how specific interpretable feature directions influence transformer outputs.

Do I need PyTorch and SAELens to use this sparse autoencoder workflow?▼

Yes, you need PyTorch (>=2.0.0), SAELens (>=6.0.0), and TransformerLens (>=2.0.0) installed. These dependencies provide the foundational environment for loading, training, and analyzing SAE activations.

How does Neuronpedia integrate with sparse autoencoder training?▼

Neuronpedia integrates with the SAELens workflow to provide end-to-end analysis of sparse autoencoder features. This integration supports loading, training, and analyzing interpretable activations discovered during mechanistic interpretability research.

Related Skills

Explore More Data & Analytics Skills→
dawsonblockdawsonblock

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced
overviewlabsoverviewlabs

sparse-autoencoder-training

Train and evaluate Sparse Autoencoders on transformer activations with SAELens and TransformerLens.

Community
Advanced
DaddyElonMusk69DaddyElonMusk69

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced
zechenzhangAGIzechenzhangAGI
11.5k

moe-training

Train Mixture-of-Experts models with DeepSpeed MoE and HuggingFace Transformers.

Community
Advanced