skills.rest
afel6
afel6COMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders on transformer activations using SAELens.

Updated Mar 25, 2026
Search Tags:#training#sae#saelens#transformer-lens#sparse-autoencoders
One-click install
npx skills add https://github.com/afel6/scal-ai-pipeline --skill sparse-autoencoder-training-afel6
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/afel6/scal-ai-pipeline/tree/main/hermes_skills_library/optional-skills/mlops/saelens
Command: npx skills add https://github.com/afel6/scal-ai-pipeline --skill sparse-autoencoder-training-afel6

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Provides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.

Core Features & Use Cases

  • Training SAEs on transformer activations (e.g., GPT-2 small) with different architectures (standard, gated, topk, jumprelu).
  • Analyzing sparsity, feature activations, and reconstruction quality to understand model internals.
  • End-to-end workflow from configuration to evaluation metrics (L0, CE loss) and ablation/steering experiments.

Quick Start

Train a configurable SAE on your target model with SAELens using LanguageModelSAERunnerConfig and run SAETrainingRunner to start training and evaluation.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train Sparse Autoencoders on transformer models to find interpretable features?▼

Train SAEs on transformer activations by configuring LanguageModelSAERunnerConfig and running SAETrainingRunner to decompose neural activations into sparse, interpretable features.

What evaluation metrics are used to measure SAE reconstruction quality?▼

SAE evaluation metrics include L0 sparsity and CE loss recovery, which analyze sparsity, feature activations, and reconstruction quality to understand model internals.

Can I use different SAE architectures like topk or gated with SAELens?▼

Yes, you can train SAEs using standard, gated, topk, and jumprelu architectures on transformer activations with SAELens.

What is the best way to analyze superposition and monosemantic representations in language models?▼

The best way to analyze superposition and monosemantic representations is to train SAEs using SAELens to decompose neural network activations into sparse, interpretable features.

Does this SAE workflow support non-destructive inference and model hooks?▼

Yes, the SAE workflow supports configurable model hooks and non-destructive inference to attach SAEs to transformer models without altering base model behavior.

Related Skills

Explore More Data & Analytics Skills→
kaminocorpkaminocorp
228

sparse-autoencoder-training

Train and evaluate Sparse Autoencoders with SAELens to analyze neural activations.

Official
Advanced
SignmanalSignmanal
1

sparse-autoencoder-training

Decompose polysemantic transformer activations into sparse interpretable features with SAELens.

Community
Advanced
JamesFincherJamesFincher

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations into sparse features.

Community
Advanced
zerwizzerwiz

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer activations with SAELens.

Community
Advanced