skills.rest
nelohenriq
nelohenriqCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Automate Sparse Autoencoder training and analysis for transformer models using SAELens.

1|Updated Mar 22, 2026
Search Tags:#superposition#mechanistic-interpretability#sae#saelens#transformer-lens#interpretable-features#activation-analysis
One-click install
npx skills add https://github.com/nelohenriq/hermes-agent-plus --skill sparse-autoencoder-training-nelohenriq
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/nelohenriq/hermes-agent-plus/tree/main/skills/mlops/evaluation/saelens
Command: npx skills add https://github.com/nelohenriq/hermes-agent-plus --skill sparse-autoencoder-training-nelohenriq

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Automates the process of training and analyzing Sparse Autoencoders (SAEs) to decompose dense model activations into sparse, interpretable features, enabling mechanistic interpretability workflows.

Core Features & Use Cases

  • Feature Discovery: identify interpretable, sparse features that respond to specific concepts in activations.
  • Superposition Analysis: study how multiple concepts share activations and how SAEs separate them.
  • Reconstruction & Evaluation: quantify reconstruction quality via L0, CE loss, and other metrics; explore steering or ablation experiments.
  • Workflow Integration: load pre-trained SAEs with SAELens, train new SAEs, and analyze features across Transformer models.

Quick Start

Train a Sparse Autoencoder on a chosen Transformer model's activations and begin analyzing the learned sparse features.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a sparse autoencoder for mechanistic interpretability?▼

You train a sparse autoencoder for mechanistic interpretability by automating the decomposition of dense transformer activations into sparse, interpretable features using this Skill's workflow.

How does a sparse autoencoder separate features in superposition?▼

A sparse autoencoder separates features in superposition by decomposing dense model activations into sparse, interpretable components, allowing you to study how multiple concepts share the same activation space.

Can I use SAELens and TransformerLens to analyze interpretable features?▼

Yes, you can use SAELens and TransformerLens to load pre-trained SAEs, train new ones, and analyze interpretable features across transformer models within this workflow.

What metrics are used for evaluating sparse autoencoder reconstruction quality?▼

Evaluating sparse autoencoder reconstruction quality involves quantifying metrics such as L0 sparsity and cross-entropy (CE) loss to measure how accurately the SAE reconstructs the original activations.

What is the best way to discover interpretable features in transformer activations?▼

The best way to discover interpretable features in transformer activations is to train a sparse autoencoder that decomposes the dense activations into sparse, concept-specific representations.

Related Skills

Explore More Data & Analytics Skills→
ever-oliever-oli
3

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer activations using SAELens.

Community
Advanced
icyzhicyzh
1

sparse-autoencoder-training

Train sparse autoencoders to decompose transformer activations into interpretable features.

Community
Advanced
gqf2008gqf2008

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations.

Community
Intermediate
InverterNetworkInverterNetwork

sparse-autoencoder-training

Trains and analyzes Sparse Autoencoders to decompose dense neural activations into sparse features.

Official
Advanced