skills.rest
Yangel-hide
Yangel-hideCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations into interpretable features.

Updated May 3, 2026
Search Tags:#training#mechanistic-interpretability#sae#transformerlens#interpretable-features#sparse-autoencoders#monosemanticity
One-click install
npx skills add https://github.com/Yangel-hide/video-production-planner-agent --skill sparse-autoencoder-training-yangel-hide
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/Yangel-hide/video-production-planner-agent/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/Yangel-hide/video-production-planner-agent --skill sparse-autoencoder-training-yangel-hide

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This section provides a practical guide for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural activations into sparse, interpretable features, supporting mechanistic interpretability research and reproducible workflows.

Core Features & Use Cases

  • Guided SAE training across architectures (Standard, Gated, TopK, JumpReLU) with configurable sparsity and training regimes.
  • Activation analysis & visualization: decode features, identify interpretable components, and relate them to model behavior.
  • End-to-end workflows: load pre-trained SAEs, train custom SAEs, and perform feature attribution, steering, and ablation.

Quick Start

Configure a LanguageModelSAERunnerConfig and run SAETrainingRunner to train an SAE on your target model.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train sparse autoencoders to find interpretable features in transformer models?▼

Train sparse autoencoders by configuring a LanguageModelSAERunnerConfig and running SAETrainingRunner via SAELens, which decomposes neural activations into sparse, interpretable features for mechanistic interpretability research.

What sparse autoencoder architectures can I use for mechanistic interpretability?▼

Sparse autoencoder training supports Standard, Gated, TopK, and JumpReLU architectures, allowing you to configure sparsity and training regimes to extract monosemantic features from transformer activations.

Do I need GPU hardware to train SAEs on GPT-2 class models?▼

Yes, training SAEs on GPT-2 class models requires compatible GPU hardware, alongside Python 3.10+, SAELens, TransformerLens, and PyTorch to process neural activations effectively.

How do I analyze and visualize activations after training a sparse autoencoder?▼

Analyze activations by decoding features, identifying interpretable components, and relating them to model behavior through feature attribution, steering, and ablation workflows provided by SAELens and TransformerLens.

Can I load pre-trained SAEs instead of training custom ones for feature discovery?▼

Yes, you can load pre-trained SAEs using SAELens to perform feature attribution, steering, and ablation directly, or train custom SAEs on your target model for specific feature discovery workflows.

What is the best way to debug transformer models using sparse autoencoders?▼

Debug transformer models by training sparse autoencoders with SAELens to decompose activations into sparse features, then applying feature attribution and ablation to isolate and analyze specific model behaviors.

Related Skills

Explore More Education & Research Skills→
dawsonblockdawsonblock

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced
icyzhicyzh
1

sparse-autoencoder-training

Train sparse autoencoders to decompose transformer activations into interpretable features.

Community
Advanced
SethyPagnaSethyPagna

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer language model activations.

Community
Advanced
KarlinskySKarlinskyS

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer language model activations.

Community
Intermediate