sparse-autoencoder-training

Name: sparse-autoencoder-training
Availability: InStock
Author: garrettroi

Train and analyze Sparse Autoencoders for neural network interpretability.

Updated Mar 12, 2026

Search Tags:#feature discovery #language models #mechanistic interpretability #neural network analysis #saelens #sparse autoencoder

Ready to run

npx skills add https://github.com/GarrettRoi/open-manus --skill sparse-autoencoder-training-garrettroi

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/GarrettRoi/open-manus/tree/main/skills/mlops/evaluation/saelens
Command: npx skills add https://github.com/GarrettRoi/open-manus --skill sparse-autoencoder-training-garrettroi

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens, transformer-lens, torch, and includes references (resource) components.

What problem does it solve?

This Skill provides guidance for training and analyzing Sparse Autoencoders (SAEs) to decompose complex neural network activations into understandable, interpretable features, addressing the challenge of polysemanticity in large language models.

Core Features & Use Cases

Feature Discovery: Identify and analyze monosemantic features within model activations.
SAE Training: Offers step-by-step instructions and configurations for training custom SAEs.
Analysis & Steering: Provides methods for analyzing individual features and steering model behavior based on discovered features.
Use Case: When investigating why a language model exhibits specific biases or generates certain types of content, use this Skill to train an SAE and analyze the features responsible for those activations.

Quick Start

Use the sparse-autoencoder-training skill to load a pre-trained SAE for GPT-2 small and analyze its features on the text "The capital of France is Paris".

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I train a sparse autoencoder for mechanistic interpretability in language models?▼

To train a sparse autoencoder for mechanistic interpretability, this skill provides step-by-step instructions and configurations to decompose dense neural network activations into sparse, interpretable features, addressing polysemanticity in large language models.

What is the best way to analyze monosemantic features in LLM activations?▼

Analyzing monosemantic features in LLM activations is best done by loading a pre-trained sparse autoencoder using sae-lens, which decomposes complex dense activations into understandable, interpretable concepts learned by the model.

How does a sparse autoencoder address polysemanticity in neural networks?▼

A sparse autoencoder addresses polysemanticity by decomposing individual neurons that react to multiple unrelated concepts into a larger set of sparse, monosemantic features, making neural network behavior interpretable.

Can I use transformer-lens and sae-lens to steer model behavior based on discovered features?▼

Yes, you can use transformer-lens and sae-lens to steer model behavior by first identifying specific responsible features through sparse autoencoder analysis, then manipulating those activations to influence the generated content.

Do I need PyTorch to load pre-trained sparse autoencoders for GPT-2?▼

Yes, you need PyTorch as a foundational dependency because both sae-lens and transformer-lens rely on torch tensors to load pre-trained sparse autoencoders and process GPT-2 dense activations.