skills.rest
MarbleSodas
MarbleSodasCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations into interpretable features.

Updated Apr 9, 2026
Search Tags:#training#interpretability#feature discovery#sae#sparse autoencoders#neuronpedia#mechanistic
One-click install
npx skills add https://github.com/MarbleSodas/Mavis --skill sparse-autoencoder-training-marblesodas
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/MarbleSodas/Mavis/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/MarbleSodas/Mavis --skill sparse-autoencoder-training-marblesodas

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens>=6.0.0, transformer-lens>=2.0.0, torch>=2.0.0, and includes references (resource) components.

What problem does it solve?

Sparse Autoencoders (SAEs) transform hard-to-read, polysemantic neural activations into sparse, more interpretable features so researchers can study and validate what a model represents.

Core Features & Use Cases

  • Feature discovery & interpretability: Identify top-activating sparse features per token to uncover candidate concepts and their activation contexts.
  • Training SAEs for mechanistic interpretability: Configure and run SAE training using SAELens to reconstruct activations with a sparsity objective.
  • Superposition and steering/attribution workflows: Analyze feature geometry (e.g., sparsity, dead features) and use feature directions for attribution and steering; validate via reconstruction and related metrics.

Quick Start

Use the skill to train or analyze a sparse autoencoder by following the provided SAELens setup, including loading a pre-trained SAE, encoding activations, and validating metrics like reconstruction quality and sparsity.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train sparse autoencoders for mechanistic interpretability?▼

Train sparse autoencoders by configuring an SAE architecture and running training loops that optimize reconstruction and sparsity losses to decompose polysemantic activations into interpretable features using SAELens.

What is feature discovery in neural networks using sparse autoencoders?▼

Feature discovery identifies candidate concepts by finding top-activating sparse features per token, revealing activation contexts to uncover what a model represents and investigate monosemanticity.

Can I analyze superposition and dead features with SAELens?▼

Yes, SAELens supports superposition analysis by evaluating feature geometry, sparsity, and dead features, enabling feature-based attribution and steering experiments for mechanistic interpretability.

Do I need TransformerLens and PyTorch to train sparse autoencoders?▼

Yes, training requires SAELens, TransformerLens, and PyTorch to load models, extract activations, and run training and evaluation loops for sparse autoencoder reconstruction.

What's the best way to evaluate sparse autoencoder reconstruction quality?▼

Evaluate reconstruction quality by loading a pre-trained SAE, encoding activations, and validating metrics like reconstruction loss and sparsity to ensure the sparse features accurately represent the model.

How does sparse autoencoder training handle polysemantic neural activations?▼

Sparse autoencoders handle polysemantic activations by applying a sparsity objective during reconstruction, transforming dense mixed signals into sparse, interpretable feature directions for analysis.

Related Skills

Explore More Education & Research Skills→
dawsonblockdawsonblock

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced
ZardLi1115ZardLi1115
4

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose model activations into interpretable features.

Community
Advanced
overviewlabsoverviewlabs

sparse-autoencoder-training

Train and evaluate Sparse Autoencoders on transformer activations with SAELens and TransformerLens.

Community
Advanced
zeronx798zeronx798

sparse-autoencoder-training

Trains and analyzes sparse autoencoders to decompose neural network activations into interpretable features.

Community
Advanced