What problem does it solve? Training sparse autoencoders (SAEs) and crosscoders for mechanistic interpretability requires wiring together model hooking, activation buffering, dictionary training, and evaluation, which is error-prone to set up from scratch. This Skill provides working scripts and guidance for the dictionary_learning library so you can train dictionaries, load pretrained weights, and compare model internals without reinventing the pipeline. ## Core Features & Use Cases - SAE Training: Train sparse autoencoders on MLP, attention, or residual stream activations using StandardTrainer, ActivationBuffer, and trainSAE with configurable sparsity penalties and dead-neuron resampling. - CrossCoder Model Diffing: Train crosscoders (including BatchTopKCrossCoder) to identify which features change between a base model and its fine-tuned variant. - Pretrained Dictionaries & Evaluation: Load pretrained AutoEncoders, JumpReLU SAEs, and crosscoders from disk or the Hugging Face Hub, and evaluate them with MSE, L0/L1 sparsity, and variance-explained metrics. - Use Case: Compare a base model against its chat fine-tuned variant by training a crosscoder on paired activations to isolate concepts introduced during fine-tuning. ## Quick Start Ask the assistant to train a sparse autoencoder on layer 1 MLP activations of Pythia-70m using the train_sae_demo.py script and report the evaluation metrics.
