skills.rest
carterwayneskhizeine
carterwayneskhizeineCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze sparse autoencoders with SAELens on TransformerLens activations.

19|4|Updated Apr 22, 2026
Search Tags:#feature discovery#sparse autoencoders#mechanistic interpretability#saelens#model steering#neuronpedia#monosemanticity
One-click install
npx skills add https://github.com/carterwayneskhizeine/hermes-agent-windows-R --skill sparse-autoencoder-training-carterwayneskhizeine
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/carterwayneskhizeine/hermes-agent-windows-R/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/carterwayneskhizeine/hermes-agent-windows-R --skill sparse-autoencoder-training-carterwayneskhizeine

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens>=6.0.0, transformer-lens>=2.0.0, torch>=2.0.0, and includes references (resource) components.

What problem does it solve?

Sparse Autoencoders often enable mechanistic interpretability, but practitioners need a reliable, repeatable workflow to train, analyze, and validate SAEs that decompose transformer activations into interpretable features.

Core Features & Use Cases

  • Train and analyze Sparse Autoencoders (SAEs) using SAELens to reconstruct model activations with a sparsity bottleneck.
  • Discover interpretable features by inspecting which sparse features activate for specific tokens, prompts, and layers.
  • Evaluate feature quality and sparsity using metrics like L0 (active features), reconstruction error/CE loss recovery, and dead-feature rates.
  • Perform feature-based experiments such as attribution, steering via decoder feature directions, and ablation-style causal testing.

Quick Start

Use this Skill to train a custom SAE by running the provided configuration template and then evaluating L0 and reconstruction/CE recovery on your target layer and dataset.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train sparse autoencoders for mechanistic interpretability using transformer activations?▼

Discover interpretable features by training sparse autoencoders with SAELens to decompose transformer activations, then inspecting which sparse features activate for specific tokens or prompts at targeted layers.

What metrics should I use to evaluate sparse autoencoder quality and sparsity?▼

Evaluate sparse autoencoder quality using L0 for active feature counts, reconstruction error for CE loss recovery, and dead-feature rates to validate the sparsity bottleneck on your target dataset.

Can I use pretrained SAEs with SAELens for feature steering and ablation?▼

Yes, you can load pretrained SAEs with SAE.from_pretrained in SAELens to perform feature-based experiments including attribution, steering via decoder feature directions, and ablation-style causal testing.

Does SAELens integrate with Neuronpedia for feature indexing?▼

SAELens supports Neuronpedia-compatible feature indexing, allowing you to map and analyze recovered sparse features within the Neuronpedia ecosystem for monosemantic representation analysis.

Do I need TransformerLens and PyTorch to train sparse autoencoders with SAELens?▼

Yes, training sparse autoencoders with SAELens requires TransformerLens for activation extraction and PyTorch, specifically needing transformer-lens>=2.0.0, sae-lens>=6.0.0, and torch>=2.0.0.

Related Skills

Explore More Education & Research Skills→
dawsonblockdawsonblock

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced
SignmanalSignmanal
1

sparse-autoencoder-training

Decompose polysemantic transformer activations into sparse interpretable features with SAELens.

Community
Advanced
Chris-Chai-MinjaeChris-Chai-Minjae

sparse-autoencoder-training

Train and analyze sparse autoencoders with SAELens and TransformerLens.

Community
Advanced
qcmuuqcmuu

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations into interpretable features.

Community
Advanced