skills.rest
Chris-Chai-Minjae
Chris-Chai-MinjaeCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze sparse autoencoders with SAELens and TransformerLens.

Updated Apr 23, 2026
Search Tags:#model-training#mechanistic-interpretability#sae#transformer-lens#sae-lens#interpretable-features#sparse-autoencoder
One-click install
npx skills add https://github.com/Chris-Chai-Minjae/hermes-agent-r1-bridge --skill sparse-autoencoder-training-chris-chai-minjae
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/Chris-Chai-Minjae/hermes-agent-r1-bridge/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/Chris-Chai-Minjae/hermes-agent-r1-bridge --skill sparse-autoencoder-training-chris-chai-minjae

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

SAELens-based training guidance helps researchers and engineers build sparse autoencoders (SAEs) that decompose dense model activations into sparse, interpretable features for mechanistic interpretability and safety research.

Core Features & Use Cases

  • Step-by-step workflows to load activations, configure SAEs, train models, and evaluate reconstruction quality.
  • Analyze sparsity patterns, feature interpretability, and cross-model consistency with SAELens integrations.
  • Use cases include understanding polysemantic activations, conducting feature-based ablations, and reproducing SAE training experiments.

Quick Start

Run a minimal SAE training workflow on a GPT-2 small layer to observe sparsity and reconstruction performance.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a sparse autoencoder to decompose transformer activations into interpretable features?▼

To train a sparse autoencoder, you configure it via LanguageModelSAERunnerConfig, load model activations, and run the training workflow to decompose dense activations into sparse, interpretable features. It tracks metrics like L0 and CE loss during the process.

What is a sparse autoencoder used for in mechanistic interpretability?▼

A sparse autoencoder is used to decompose dense, polysemantic model activations into interpretable features for mechanistic interpretability and safety research. This helps researchers understand individual feature representations within transformer models.

Can I load and evaluate pre-trained SAEs instead of training a new one?▼

Yes, you can load pre-trained SAEs to evaluate their sparsity patterns, reconstruction quality, and feature interpretability. This allows you to perform feature attribution analyses across transformer models without training from scratch.

Does SAELens work with TransformerLens for reproducing SAE training experiments?▼

Yes, SAELens integrates with TransformerLens to provide reproducible experimentation environments for sparse autoencoders. You can run minimal SAE training workflows on models like GPT-2 small to observe sparsity and reconstruction performance.

How do I evaluate sparsity and reconstruction quality when training SAEs?▼

You evaluate sparsity and reconstruction quality by tracking metrics like L0 and CE loss during SAE training. The workflow allows you to analyze sparsity patterns and cross-model consistency to ensure the interpretable features are valid.

Related Skills

Explore More Data & Analytics Skills→
GregsGreyCodeGregsGreyCode
6

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer activations using SAELens.

Community
Advanced
helix4uhelix4u

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose neural network activations into interpretable features.

Community
Advanced
dawsonblockdawsonblock

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced
MarbleSodasMarbleSodas

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations into interpretable features.

Community
Intermediate