skills.rest
ever-oli
ever-oliCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer activations using SAELens.

3|Updated Mar 20, 2026
Search Tags:#model analysis#sae#mechanistic interpretability#saelens#sparse autoencoder#interpretable features#transformer activations
One-click install
npx skills add https://github.com/ever-oli/io --skill sparse-autoencoder-training-ever-oli
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/ever-oli/io/tree/main/skills/mlops/evaluation/saelens
Command: npx skills add https://github.com/ever-oli/io --skill sparse-autoencoder-training-ever-oli

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens>=6.0.0, transformer-lens>=2.0.0, torch>=2.0.0, and includes references (resource) components.

What problem does it solve?

SAELens provides a streamlined workflow to train and analyze Sparse Autoencoders (SAEs) that decompose neural activations into sparse, interpretable features, enabling mechanistic interpretability research.

Core Features & Use Cases

  • Load and evaluate pre-trained SAEs on model activations to identify interpretable features.
  • Train custom SAEs to study monosemantic representations and feature discovery in language models.
  • Analyze activation reconstructions and perform feature steering and ablation across prompts.

Quick Start

Load a model, load a pretrained SAE, encode activations, and inspect the top features for a sample prompt.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a sparse autoencoder on transformer activations?▼

Train a sparse autoencoder on transformer activations using SAELens to decompose complex neural activations into modular, interpretable features for mechanistic interpretability research.

What is mechanistic interpretability and how do sparse autoencoders help?▼

Mechanistic interpretability uses sparse autoencoders to break down dense transformer activations into sparse, monosemantic features, making neural network representations modular and interpretable.

Can I load and evaluate pre-trained SAEs on language model activations?▼

Load and evaluate pre-trained SAEs on language model activations using SAELens to identify interpretable features and analyze activation reconstructions across sample prompts.

Do I need a GPU to train sparse autoencoders with SAELens?▼

GPU access is strongly recommended to train sparse autoencoders with SAELens, though the workflow also requires Python 3.x, TransformerLens, and PyTorch.

How do I perform feature steering and ablation on transformer activations?▼

Perform feature steering and ablation on transformer activations by encoding activations with SAELens, identifying top interpretable features, and modifying them across prompts.

What is the best way to analyze monosemantic representations in language models?▼

Analyze monosemantic representations in language models by training custom sparse autoencoders with SAELens to discover and isolate interpretable features within dense activations.

Related Skills

Explore More Data & Analytics Skills→
vikrant-projectvikrant-project
2

sparse-autoencoder-training

Train Sparse Autoencoders on neural network activations using Python libraries.

Community
Intermediate
RheasilviaRheasilvia

sparse-autoencoder-training

Train Sparse Autoencoders to decompose transformer activations into interpretable features.

Community
Advanced
arsityarsity

sparse-autoencoder-training

Train Sparse Autoencoders on neural activations with TransformerLens via SAELens APIs.

Community
Advanced
icyzhicyzh
1

sparse-autoencoder-training

Train sparse autoencoders to decompose transformer activations into interpretable features.

Community
Advanced