skills.rest
cesaregarza
cesaregarzaCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

mechinterp-decoder

Analyze SAE decoder weights to identify promoted outputs and cluster features.

1|Updated Jul 9, 2024
Search Tags:#weights#analyze#decoder#SAE#feature-influence#decoder-similarity
One-click install
npx skills add https://github.com/cesaregarza/SplatNLP --skill mechinterp-decoder
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mechinterp-decoder
Source: https://github.com/cesaregarza/SplatNLP/tree/main/.claude/skills/mechinterp-decoder
Command: npx skills add https://github.com/cesaregarza/SplatNLP --skill mechinterp-decoder

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps researchers interpret what a given SAE decoder feature actually promotes, by analyzing the decoder weights rather than only activation patterns. It shifts focus from activation hotspots to downstream output tendencies.

Core Features & Use Cases

  • Output Influence: identify tokens the feature tends to promote or suppress.
  • Decoder Similarity: cluster features by similar output patterns to discover shared roles.
  • Experiment Workflow: integrate with mechinterp investigation to guide hypothesis formation and labeling.

Quick Start

Install dependencies for the mechinterp tooling and run the decoder CLI to view output influence:

  • Basic query: poetry run python -m splatnlp.mechinterp.cli.decoder_cli output-influence --feature-id 13934 --model ultra
  • JSON output: poetry run python -m splatnlp.mechinterp.cli.decoder_cli output-influence --feature-id 13934 --model ultra --format json
  • Explore similar features: poetry run python -m splatnlp.mechinterp.cli.decoder_cli similar --feature-id 13934 --model ultra --top-k 10

Frequently Asked Questions about mechinterp-decoder

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I analyze SAE decoder weights to see what tokens a feature promotes?▼

Analyze SAE decoder weights by running a decoder CLI to compute output influence, querying specific feature IDs against model checkpoints to identify which tokens the feature promotes or suppresses.

What is the difference between analyzing SAE activation patterns and decoder weights?▼

Analyzing SAE decoder weights reveals downstream output tendencies a feature promotes, shifting focus from activation hotspots to output influence, whereas activation patterns only show where a feature triggers.

How do I cluster SAE features by similar output patterns?▼

Cluster SAE features by decoder similarity using a decoder CLI's similar command, specifying a feature ID and model checkpoint to retrieve top-k features with shared output roles and patterns.

Can I compare SAE feature output tendencies across different models and AP levels?▼

Yes, you can compare output tendencies across models and AP levels by computing decoder weight percentiles and similarity metrics for SAE features using model checkpoints like ultra.

Do I need specific model checkpoints to compute SAE decoder output influence?▼

Yes, you need model checkpoints, such as the ultra model, and the mechinterp decoder tooling installed to compute output influence, decoder weight percentiles, and similarity metrics.

How do I export SAE decoder output influence results to JSON?▼

Export SAE decoder output influence to JSON by appending the --format json flag to the decoder CLI command when querying a feature ID and model checkpoint.

Related Skills

Explore More Data & Analytics Skills→
bianhaifeng789-huebianhaifeng789-hue

write-compressor

Implement deterministic compressor encoders compatible with existing decoders.

Community
Advanced
Gull-StackGull-Stack

ab-test-setup

Designs, plans, and analyzes A/B tests including hypothesis, metrics, and sample size calculations.

Official
Advanced
jeffdevillejeffdeville

Example Skill

Automate skill activation when users mention trigger phrases in conversation.

Community
Basic
KarlinskySKarlinskyS

sparse-autoencoder-training

Train and analyze sparse autoencoders on transformer language model activations.

Community
Intermediate