skills.rest
Monjyu1101
Monjyu1101COMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders on transformer activations using SAELens workflows.

1|Updated Jan 31, 2026
Search Tags:#training#transformers#feature-discovery#mechanistic-interpretability#sae#saelens#sparse-autoencoder
One-click install
npx skills add https://github.com/Monjyu1101/AiDiy2026 --skill sparse-autoencoder-training-monjyu1101
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/Monjyu1101/AiDiy2026/tree/main/backend_hermes/optional-skills/mlops/saelens
Command: npx skills add https://github.com/Monjyu1101/AiDiy2026 --skill sparse-autoencoder-training-monjyu1101

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens>=6.0.0, transformer-lens>=2.0.0, torch>=2.0.0, and includes references (resource) components.

What problem does it solve?

Sparse Autoencoders (SAEs) are used to decompose dense model activations into sparse, interpretable features, enabling mechanistic interpretability studies and targeted analysis of transformer activations.

Core Features & Use Cases

  • Train SAEs on transformer activations to discover monosemantic features and reduce polysemanticity.
  • Evaluate sparsity control, reconstruction quality, and feature attributions across models using SAELens.
  • Use SAELens workflows to load pretrained SAEs, train new instances, and perform feature attribution and steering analyses.

Quick Start

Load a pretrained SAE and begin analyzing its learned sparse features on model activations.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I train sparse autoencoders for mechanistic interpretability in transformer models?▼

To train sparse autoencoders for mechanistic interpretability, this Skill configures model names, hook points, SAE architecture, and sparsity parameters within SAELens workflows to decompose dense transformer activations into interpretable features.

What is the best way to reduce polysemanticity when discovering features in transformer activations?▼

Reducing polysemanticity during feature discovery involves training sparse autoencoders on transformer activations to extract monosemantic features, enabling targeted analysis and improved reconstruction quality evaluation.

Can I load pretrained SAEs and perform feature attribution using SAELens?▼

Yes, you can load pretrained SAEs using SAELens workflows to perform feature attribution and steering analyses on transformer model activations, evaluating learned sparse features without training new instances.

Do I need PyTorch and TransformerLens installed to configure sparsity parameters for SAEs?▼

Yes, PyTorch, TransformerLens, and SAELens are required dependencies to configure sparsity parameters, evaluate reconstruction quality, and execute feature discovery workflows across transformer models.

How does sparsity control affect reconstruction quality when evaluating sparse autoencoders?▼

Sparsity control directly impacts reconstruction quality by constraining the activation bottleneck, allowing researchers to evaluate the trade-off between feature sparsity and dense activation reconstruction fidelity across transformer models.

What are the limitations of using sparse autoencoders for feature discovery in transformer models?▼

Sparse autoencoders for feature discovery require careful hook point selection and sparsity parameter tuning, and may struggle with reconstruction quality if architecture configurations do not match the target transformer model's activation dimensions.

Related Skills

Explore More Data & Analytics Skills→
SethyPagnaSethyPagna

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer language model activations.

Community
Advanced
SignmanalSignmanal
1

sparse-autoencoder-training

Decompose polysemantic transformer activations into sparse interpretable features with SAELens.

Community
Advanced
overviewlabsoverviewlabs

sparse-autoencoder-training

Train and evaluate Sparse Autoencoders on transformer activations with SAELens and TransformerLens.

Community
Advanced
cloudlinesscloudliness

sparse-autoencoder-training

Train and analyze Sparse Autoencoders with SAELens on transformer activations.

Community
Advanced