skills.rest
zangjeicy
zangjeicyCOMMUNITY
Explore all skills by this author →
GitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose neural network activations into interpretable features.

Updated Jul 13, 2026
Search Tags:#neural networks#superposition#feature discovery#sae#sparse autoencoders#mechanistic interpretability
One-click install
npx skills add https://github.com/zangjeicy/Hermes --skill sparse-autoencoder-training-zangjeicy
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/zangjeicy/Hermes/tree/main/optional-skills/mlops/saelens
Command: npx skills add https://github.com/zangjeicy/Hermes --skill sparse-autoencoder-training-zangjeicy

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens, transformer-lens, torch, and includes references (resource) components.

What problem does it solve?

This skill addresses the challenge of polysemanticity in neural networks, where individual neurons represent multiple, overlapping concepts, making model behavior difficult to interpret.

Core Features & Use Cases

  • Feature Discovery: Decompose dense model activations into sparse, monosemantic features using Sparse Autoencoders (SAEs).
  • Mechanistic Interpretability: Analyze superposition and feature geometry to understand what concepts a model has learned.
  • Use Case: Researchers can use this to identify safety-relevant features like deception or bias within a language model's internal activations.

Quick Start

Use the sparse-autoencoder-training skill to load a pre-trained SAE for gpt2-small and identify the top-activating features for a given input prompt.

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What is a sparse autoencoder used for in neural networks?▼

A sparse autoencoder decomposes dense neural network activations into interpretable, monosemantic features to resolve polysemanticity, allowing researchers to identify distinct concepts a model has learned.

How do I train sparse autoencoders to find interpretable features in language models?▼

You train sparse autoencoders by integrating the sae-lens and transformer-lens libraries to manipulate model activations, perform feature discovery, and run reconstruction validation to isolate safety-relevant features.

Why do neurons in neural networks represent multiple concepts and how can I separate them?▼

Neurons exhibit polysemanticity due to superposition, representing overlapping concepts. You can separate them by training sparse autoencoders to map these dense activations into sparse, monosemantic feature spaces.

Can I use transformer-lens and sae-lens to analyze feature geometry in language models?▼

Yes, you can use transformer-lens and sae-lens to analyze feature geometry and superposition. These libraries support loading models like gpt2-small to inspect top-activating features for specific prompts.

What is the best way to identify safety-relevant features like deception in a language model?▼

The best way to identify safety-relevant features is by training sparse autoencoders to decompose internal activations, isolating specific concepts like deception or bias for mechanistic interpretability analysis.

Do I need pre-trained models to perform feature discovery and feature-based steering?▼

You need a compatible language model like gpt2-small to extract activations. The workflow involves loading a pre-trained sparse autoencoder or training one to validate reconstruction and perform feature-based ablation.

Related Skills

Explore More Education & Research Skills→
heysuhasheysuhas
2

sparse-autoencoder-training

Decompose dense neural network activations into sparse, interpretable features.

Community
Advanced
SignmanalSignmanal
1

sparse-autoencoder-training

Decompose polysemantic transformer activations into sparse interpretable features with SAELens.

Community
Advanced
zechenzhangAGIzechenzhangAGI
11.5k

moe-training

Train Mixture-of-Experts models with DeepSpeed MoE and HuggingFace Transformers.

Community
Advanced
pacphipacphi
6

flow-nexus-neural

Trains and deploys neural networks in distributed E2B sandbox environments.

Community
Advanced