skills.rest
ZardLi1115
ZardLi1115COMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose model activations into interpretable features.

4|Updated May 18, 2026
Search Tags:#training#superposition#interpretability#feature discovery#sparse autoencoders#saelens#neuron analysis
One-click install
npx skills add https://github.com/ZardLi1115/zedclaw --skill sparse-autoencoder-training-zardli1115
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/ZardLi1115/zedclaw/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/ZardLi1115/zedclaw --skill sparse-autoencoder-training-zardli1115

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens>=6.0.0, transformer-lens>=2.0.0, torch>=2.0.0, and includes references (resource) components.

What problem does it solve?

This Skill helps you train and analyze Sparse Autoencoders (SAEs) to decompose polysemantic neural activations into sparse, interpretable features.

Core Features & Use Cases

  • Train Custom SAEs: Configure and run SAE training with controllable sparsity via L1 regularization or top-k/gated variants.
  • Load & Analyze Pretrained SAEs: Encode activations, inspect feature activation patterns, and evaluate reconstruction quality (e.g., CE recovery).
  • Interpretability Workflows: Support feature discovery, superposition analysis, monosemantic feature study, and feature steering/attribution experiments.

Quick Start

Instruct the agent to load a pretrained SAE for a chosen layer, encode activations for a prompt, then report the top activating features and reconstruction error.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train a sparse autoencoder to find interpretable features in neural network activations?▼

Load a pretrained SAE for a chosen layer using SAELens, encode the activation vectors, and report the top activating features alongside the reconstruction error. This workflow reveals which specific neurons fire for a given input prompt.

Can I analyze superposition and geometry in model activations using sparse autoencoders?▼

Yes, you can use sparse autoencoders to analyze superposition and geometry in model activations by applying feature discovery workflows and inspecting monosemantic representation patterns. This reveals how neural networks encode overlapping concepts.

Do I need SAELens and TransformerLens to run feature discovery and steering experiments?▼

Yes, you need SAELens for SAE training and evaluation, TransformerLens for model activations, and torch for the underlying tensor operations. These dependencies are required to run feature discovery and steering experiments effectively.

What's the best way to evaluate sparse autoencoder reconstruction quality?▼

Evaluate sparse autoencoder reconstruction quality by monitoring sparsity metrics and calculating cross-entropy (CE) recovery. This directly measures how well the SAE reconstructs the original model activations without losing performance.

How do I configure L1 regularization and sparsity when training sparse autoencoders?▼

Configure sparsity during sparse autoencoder training by adjusting the L1 regularization penalty or selecting top-k and gated variants. This controls how strictly the model limits active features during the encoding and decoding process.

Related Skills

Explore More Education & Research Skills→
icyzhicyzh
1

sparse-autoencoder-training

Train sparse autoencoders to decompose transformer activations into interpretable features.

Community
Advanced
cxnaivecxnaive

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose dense neural activations into sparse features.

Community
Advanced
tianhao909tianhao909
1

transformer-lens-interpretability

Inspect and manipulate transformer internal activations and weights for mechanistic interpretability.

Community
Advanced
MarbleSodasMarbleSodas

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens to decompose neural activations into interpretable features.

Community
Intermediate