skills.rest
tadod12
tadod12COMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Automates SAE training/evaluation to decompose model activations into interpretable features using SAELens metrics.

Updated Mar 18, 2026
Search Tags:#model-training#mechanistic-interpretability#sae#sae-lens#sparse-autoencoders#activation-analysis
One-click install
npx skills add https://github.com/tadod12/fraud-detection-research --skill sparse-autoencoder-training-tadod12
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/tadod12/fraud-detection-research/tree/main/.agent/skills/04-mechanistic-interpretability/saelens
Command: npx skills add https://github.com/tadod12/fraud-detection-research --skill sparse-autoencoder-training-tadod12

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Provides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.

Core Features & Use Cases

  • Guidance for training Sparse Autoencoders (SAEs) with SAELens to decompose activations into sparse, interpretable features.
  • Workflows for loading pre-trained SAEs, training from scratch, and performing feature analysis, steering, and ablation to study monosemantic representations.
  • Real-world use cases include discovering interpretable features, studying superposition, and conducting feature-based interventions in language models.

Quick Start

Run the training workflow to train an SAE on a chosen model and evaluate L0, CE, and sparsity metrics.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train sparse autoencoders to extract interpretable features from language model activations?▼

To train sparse autoencoders for interpretable features, you can automate the SAE training process to decompose model activations into sparse representations. This includes configuring the architecture, setting sparsity controls, and evaluating reproducible metrics like L0 and CE loss.

What is the best way to analyze superposition in neural networks using sparse autoencoders?▼

Analyzing superposition with sparse autoencoders involves decomposing model activations into sparse, monosemantic representations. Researchers use this technique to reveal interpretable features and study how neural networks represent overlapping concepts.

Can I load pre-trained SAEs for feature analysis and steering experiments?▼

Yes, you can load pre-trained SAEs to perform feature analysis, steering, and ablation. This allows you to study monosemantic representations and conduct feature-based interventions in language models without training from scratch.

What evaluation metrics are used when training sparse autoencoders for mechanistic interpretability?▼

When training sparse autoencoders for mechanistic interpretability, the primary evaluation metrics include L0 sparsity, cross-entropy (CE) loss, and overall sparsity measures. These metrics ensure the SAE effectively decomposes activations into interpretable features.

How do I conduct feature-based interventions in language models after training an SAE?▼

To conduct feature-based interventions in language models, you use the trained SAE to perform steering and ablation on the extracted interpretable features. This reveals how specific monosemantic representations influence model behavior.

Related Skills

Explore More Education & Research Skills→
Human-Frontier-Labs-IncHuman-Frontier-Labs-Inc

flow-nexus-neural

Train and deploy neural networks across distributed E2B sandboxes.

Official
Advanced
cxnaivecxnaive

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose dense neural activations into sparse features.

Community
Advanced
pacphipacphi
14

PyTorch ML

Define and train PyTorch neural networks with CUDA acceleration.

Community
Advanced
arsityarsity

sparse-autoencoder-training

Train Sparse Autoencoders on neural activations with TransformerLens via SAELens APIs.

Community
Advanced