sparse-autoencoder-training

You can train a sparse autoencoder by configuring a LanguageModelSAERunnerConfig and executing SAETrainingRunner to decompose neural activations into sparse, interpretable features from your model's residual stream.

Supported sparse autoencoder architectures include Standard, Gated, TopK, and JumpReLU. You can adjust parameters like d_in, d_sae, activation functions, and training hyperparameters to study monosemantic representations in language models.

Yes, the skill provides end-to-end workflows for feature attribution, steering, and ablation using TransformerLens models to evaluate monosemantic representations and study superposition in language model activations.

A sparse autoencoder decomposes dense neural activations into a higher-dimensional sparse representation, forcing individual features to align with interpretable concepts and reducing superposition in language models.

The skill provides evaluation metrics to assess the quality of trained sparse autoencoders, helping you measure monosemanticity and feature interpretability when analyzing GPT-2 residual stream activations.

Yes, configuring expansion factors is a core feature. You adjust d_in and d_sae dimensions alongside training hyperparameters to control the capacity of the sparse autoencoder for discovering interpretable features.