skills.rest
Rawgrowth-Consulting
Rawgrowth-ConsultingOFFICIAL
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations into interpretable features.

Updated Apr 23, 2026
Search Tags:#training#superposition#feature-discovery#mechanistic-interpretability#sae#saelens#sparse-autoencoders
One-click install
npx skills add https://github.com/Rawgrowth-Consulting/rawclaw-agent --skill sparse-autoencoder-training-rawgrowth-consulting
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/Rawgrowth-Consulting/rawclaw-agent/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/Rawgrowth-Consulting/rawclaw-agent --skill sparse-autoencoder-training-rawgrowth-consulting

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Guides researchers in training and analyzing Sparse Autoencoders (SAEs) with SAELens to decompose dense neural activations into sparse, interpretable features for mechanistic interpretability.

Core Features & Use Cases

  • End-to-end guidance for configuring and training SAEs across architectures (Standard, Gated, TopK, JumpReLU) using LanguageModelSAERunnerConfig.
  • Tools for analyzing SAE activations, identifying top features per token, and evaluating reconstruction quality.
  • Use cases include mechanistic interpretability research, feature discovery, and understanding superposition in language models.

Quick Start

Install SAELens, choose a base model and hook, configure LanguageModelSAERunnerConfig, and run the SAE training workflow.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train sparse autoencoders to decompose transformer model activations into interpretable features?▼

To train sparse autoencoders, you configure a LanguageModelSAERunnerConfig with your chosen base model and hook point, then run the SAE training workflow to decompose dense neural activations into sparse, interpretable features for mechanistic interpretability research.

What sparse autoencoder architectures are supported in SAELens for mechanistic interpretability research?▼

SAELens supports standard, gated, top-k, and JumpReLU sparse autoencoder architectures. You can configure these SAE architectures using the LanguageModelSAERunnerConfig to analyze neural activations and understand superposition in transformer models.

How do I analyze SAE activations and identify top features per token in language models?▼

You analyze SAE activations by evaluating reconstruction quality and identifying top features per token after training. This workflow helps decompose dense neural activations into sparse, interpretable features, enabling feature discovery and understanding of superposition in language models.

Do I need TransformerLens and SAELens to start training sparse autoencoders for feature discovery?▼

Yes, you need to install SAELens and TransformerLens to train sparse autoencoders. You select a base model and hook point, configure the LanguageModelSAERunnerConfig, and run the workflow to decompose dense neural activations into sparse, interpretable features.

What is the best way to understand superposition in language models using sparse autoencoders?▼

The best way to understand superposition is to train sparse autoencoders using SAELens to decompose dense neural activations into sparse, interpretable features. This mechanistic interpretability approach reveals how language models represent overlapping features within dense layers.

How do I evaluate reconstruction quality when analyzing sparse autoencoders with SAELens?▼

You evaluate reconstruction quality by analyzing the trained SAE activations against the base model's dense activations. This analysis identifies top features per token and verifies how well the sparse autoencoder decomposes neural activations into interpretable components.

Related Skills

Explore More Education & Research Skills→
cloudlinesscloudliness

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens on transformer activations.

Community
Advanced
davpatel605-beepdavpatel605-beep

sparse-autoencoder-training

Train sparse autoencoders to decompose transformer activations into interpretable features.

Community
Advanced
Devsoul2026Devsoul2026
150

sparse-autoencoder-training

Train sparse autoencoders on transformer activations using SAELens and TransformerLens.

Community
Advanced
RheasilviaRheasilvia

sparse-autoencoder-training

Train Sparse Autoencoders to decompose transformer activations into interpretable features.

Community
Advanced