sparse-autoencoder-training

Train and analyze sparse autoencoders to interpret neural network activations.

2|1|Updated May 10, 2026

Search Tags:#deep learning #interpretability #autoencoder #activations #mechanistic #neural features

Ready to run

npx skills add https://github.com/zli5460/hermes-agent-X-Phoenix-Architecture --skill sparse-autoencoder-training-zli5460

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/zli5460/hermes-agent-X-Phoenix-Architecture/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/zli5460/hermes-agent-X-Phoenix-Architecture --skill sparse-autoencoder-training-zli5460

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens, transformer-lens, torch, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill facilitates training and analyzing Sparse Autoencoders (SAEs) to interpret neural network activations, helping researchers uncover meaningful features and understand model polysemanticity.

Core Features & Use Cases

Feature Discovery: Train SAEs to decompose activations into sparse, interpretable components, aiding in mechanistic interpretability studies.
Activation Analysis: Examine model behaviors such as superposition, polysemantic neurons, and feature attribution in language models.
Use Case: A researcher wants to analyze GPT-2 activations to identify features representing grammatical structures. They can train a SAE and interpret the resulting features.

Quick Start

Load a pre-trained SAE for GPT-2 and analyze which features activate on specific tokens.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I train a sparse autoencoder to find interpretable features in neural network activations?▼

To train a sparse autoencoder for interpretable features, you use this Skill to decompose neural network activations into sparse components. It supports training routines to uncover meaningful features and analyze model polysemanticity.

Can I use SAELens to analyze GPT-2 activations for grammatical structures?▼

Yes, you can use SAELens with TransformerLens to analyze GPT-2 activations for grammatical structures. The Skill enables you to train a sparse autoencoder and interpret which features activate on specific tokens.

What is the best way to interpret polysemantic neurons in language models?▼

The best way to interpret polysemantic neurons is training sparse autoencoders to decompose activations. This Skill helps you examine superposition and feature attribution to resolve polysemanticity in language models.

Does this Skill require transformer-lens and torch to run sparse autoencoder training?▼

Yes, this Skill requires transformer-lens and torch as dependencies. It leverages these frameworks alongside SAELens to execute custom training routines and analyze neural network activations.

How do I load a pre-trained SAE to analyze feature activation on specific tokens?▼

You load a pre-trained SAE for models like GPT-2 using the provided scripts. This allows you to quickly analyze and identify which interpretable neural features activate on specific input tokens.

Why do I need sparse autoencoders for mechanistic interpretability research?▼

You need sparse autoencoders for mechanistic interpretability because they decompose complex activations into interpretable components. This resolves polysemanticity and helps uncover meaningful features representing model behaviors.