skills.rest
DoanNgocCuong
DoanNgocCuongCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

sparse-autoencoder-training

Train and analyze Sparse Autoencoders using the SAELens library.

Updated Aug 27, 2026
Search Tags:#llm analysis#interpretability#feature discovery#sae#mechanistic interpretability#sparse autoencoder
One-click install
npx skills add https://github.com/DoanNgocCuong/continuous-training-pipeline_T3_2026 --skill sparse-autoencoder-training-doanngoccuong
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sparse-autoencoder-training
Source: https://github.com/DoanNgocCuong/continuous-training-pipeline_T3_2026/tree/main/.claude/skills/saelens
Command: npx skills add https://github.com/DoanNgocCuong/continuous-training-pipeline_T3_2026 --skill sparse-autoencoder-training-doanngoccuong

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sae-lens, transformer-lens, torch, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps you understand the internal workings of large language models by decomposing their complex activations into simpler, interpretable features using Sparse Autoencoders (SAEs).

Core Features & Use Cases

  • Feature Discovery: Uncover the concepts and patterns learned by neural networks.
  • Mechanistic Interpretability: Analyze how models represent information, including phenomena like superposition.
  • Use Case: You can use this Skill to find out what specific features in a language model activate when it processes legal jargon, or to understand how a model represents different emotions in text.

Quick Start

Use the saelens skill to load a pre-trained SAE for GPT-2 small and encode activations from the text "The capital of France is Paris".

Frequently Asked Questions about sparse-autoencoder-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find interpretable features in LLM activations using sparse autoencoders?▼

To find interpretable features in LLM activations, you train a sparse autoencoder using the SAELens library to decompose complex neural network activations into simpler, monosemantic concepts. This allows you to uncover specific patterns learned by the model.

What is mechanistic interpretability and how do sparse autoencoders help with it?▼

Mechanistic interpretability analyzes how neural networks internally represent information. Sparse autoencoders help by decomposing complex activations into interpretable features, allowing you to study phenomena like superposition and understand exactly how models process text.

Can I load a pre-trained SAE for GPT-2 instead of training a custom one from scratch?▼

Yes, you can load a pre-trained SAE for models like GPT-2 small using the SAELens library. This allows you to immediately encode activations and analyze model representations without spending time and compute resources training a custom autoencoder.

How do I train a custom sparse autoencoder with configurable hyperparameters?▼

You can train a custom sparse autoencoder by configuring hyperparameters and evaluation metrics through the provided SAELens tools. This supports mechanistic interpretability research by allowing you to tailor the feature discovery process to specific model layers or datasets.

Do I need PyTorch and TransformerLens to analyze LLM internals with SAELens?▼

Yes, you need PyTorch and TransformerLens along with the SAELens library. These dependencies provide the necessary framework to extract neural network activations and train or load sparse autoencoders for decomposition and feature analysis.

What is the best way to understand how a language model represents legal jargon or emotions?▼

The best way is to use SAELens to train or load a sparse autoencoder and extract feature activations from the target text. This decomposition reveals the specific interpretable, monosemantic features that fire when the model processes legal or emotional context.

Related Skills

Explore More Education & Research Skills→
dawsonblockdawsonblock

sparse-autoencoder-training

Train sparse autoencoders on transformer activations with SAELens.

Community
Advanced
ysyecustysyecust
40

numerical-patterns

Implement C++20 numerical patterns for dense and sparse matrix operations.

Community
Advanced
GregsGreyCodeGregsGreyCode
6

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer activations using SAELens.

Community
Advanced
OpenLAIROpenLAIR
1.0k

transformer-lens-interpretability

Analyze and steer transformer activations with TransformerLens for mechanistic interpretability.

Official
Advanced