Verified:GitHub

sparse-autoencoder-training

Train Sparse Autoencoders to decompose transformer activations into interpretable features.

Updated Jun 25, 2026

Search Tags:#neural networks #feature discovery #sae #sparse autoencoders #mechanistic interpretability #transformer-lens

Ready to run

npx skills add https://github.com/Rheasilvia/hermes-desktop --skill sparse-autoencoder-training-rheasilvia

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/Rheasilvia/hermes-desktop/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/Rheasilvia/hermes-desktop --skill sparse-autoencoder-training-rheasilvia

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens, transformer-lens, torch, and includes references (resource) components.

What problem does it solve?

This skill addresses the challenge of polysemanticity in neural networks, where individual neurons activate for multiple, unrelated concepts, making model behavior opaque and difficult to interpret.

Core Features & Use Cases

Feature Discovery: Decompose dense model activations into sparse, monosemantic features using Sparse Autoencoders (SAEs).
Model Steering: Identify and manipulate specific features to influence model output, such as steering a model toward or away from specific concepts.
Interpretability Analysis: Use pre-trained SAEs to analyze model behavior, perform feature attribution, and validate reconstruction quality.

Quick Start

Use the sparse-autoencoder-training skill to load a pre-trained SAE for gpt2-small and identify the top-activating features for a given input prompt.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I decompose dense neural network activations into interpretable features?▼

To decompose dense neural network activations into interpretable features, train Sparse Autoencoders (SAEs) that map dense activations into sparse, monosemantic vectors, resolving polysemanticity and making model behavior easier to interpret.

What is polysemanticity in neural networks and how do sparse autoencoders resolve it?▼

Polysemanticity in neural networks occurs when individual neurons activate for multiple unrelated concepts. Sparse autoencoders resolve this by decomposing dense activations into sparse, monosemantic features, isolating distinct concepts for mechanistic interpretability.

Can I use transformer-lens with sae-lens to identify and manipulate model features for steering?▼

Yes, integrating transformer-lens with sae-lens enables feature discovery and model steering. You can load pre-trained SAEs to identify top-activating features for a given prompt and manipulate them to influence model output toward or away from specific concepts.

How do I find top-activating features for a given prompt in gpt2-small using sparse autoencoders?▼

To find top-activating features in gpt2-small, use this skill to load a pre-trained SAE via sae-lens and transformer-lens, pass your input prompt through the model, and extract the highest-activating sparse features from the encoded activations.

Do I need pre-trained sparse autoencoders to perform feature attribution and ablation analysis?▼

You do not need to train from scratch to perform feature attribution and ablation analysis. You can use pre-trained sparse autoencoders to analyze model behavior, perform feature attribution, and validate reconstruction quality across supported transformer architectures.

What are the limitations of using sparse autoencoders for mechanistic interpretability?▼

Sparse autoencoders for mechanistic interpretability require integration with transformer-lens and sae-lens to perform activation encoding and reconstruction. They depend on reconstruction quality validation and may not fully resolve all polysemantic neurons across complex transformer architectures.