skills.rest
AlexiosBluffMara
AlexiosBluffMaraCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze sparse autoencoders with SAELens for transformer interpretability.

2|Updated Apr 25, 2026
Search Tags:#feature-discovery#sae#sae-lens#interpretable-features#sparse-autoencoder#transformer-activations#training-tutorials
One-click install
npx skills add https://github.com/AlexiosBluffMara/mercury --skill sparse-autoencoder-training-alexiosbluffmara
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/AlexiosBluffMara/mercury/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/AlexiosBluffMara/mercury --skill sparse-autoencoder-training-alexiosbluffmara

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Provides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.

Core Features & Use Cases

  • Guidance on training SAEs with SAELens across transformer activations and various architectures.
  • Methods to analyze learned features, sparsity metrics, and reconstruction quality.
  • Use Case: Build interpretable feature representations from model activations to inspect polysemanticity.

Quick Start

Run an end-to-end SAE workflow to encode transformer activations and inspect top features across layers.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a sparse autoencoder on transformer activations using SAELens?▼

To train a sparse autoencoder with SAELens, you need a compatible TransformerLens model and activation data to run an end-to-end workflow across various architectures. This process encodes transformer activations into interpretable features for mechanistic interpretability research.

What is a sparse autoencoder used for in mechanistic interpretability?▼

A sparse autoencoder decomposes neural network activations into interpretable features to study monosemantic representations and superposition in language models. It helps discover interpretable features by resolving polysemanticity in transformer activation datasets.

Can I evaluate sparsity metrics and reconstruction quality for SAEs?▼

Yes, you can evaluate sparsity metrics and reconstruction quality for SAEs trained with SAELens. The workflow includes methods to analyze learned features, measure sparsity, and assess reconstruction quality across transformer activation datasets.

Does training sparse autoencoders with SAELens require specific dependencies?▼

Training sparse autoencoders with SAELens requires a compatible TransformerLens model and suitable activation data to run end-to-end experiments. These components are necessary to build interpretable feature representations from model activations.

How do I perform feature analysis and steering after training an SAE?▼

After training an SAE with SAELens, you can perform feature analysis and steering by inspecting top features across layers to analyze learned representations. This allows you to evaluate polysemanticity and apply steering across activation datasets.

When should I use sparse autoencoders for feature discovery in language models?▼

Use sparse autoencoders when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models. They are specifically needed to inspect polysemanticity and build interpretable feature representations from model activations.

Related Skills

Explore More Education & Research Skills→
zli5460zli5460
2

sparse-autoencoder-training

Train and analyze sparse autoencoders to interpret neural network activations.

Community
Intermediate
NousResearchNousResearch
227.8k

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens on GPT-2 activations.

Official
Advanced
dawsonblockdawsonblock

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced
Tommy-ywTommy-yw
539

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose neural network activations into interpretable features.

Community
Advanced