What problem does it solve? Interpreting large language models requires finding which internal components matter most, but training probes or running backward-pass attribution is expensive. Magnitude Analysis provides a lightweight first-pass screening method that ranks weights, neurons, and layer representations by numerical magnitude to surface candidate components for deeper causal verification. ## Core Features & Use Cases - Static Parameter Analysis: Compute per-weight or per-row norms of weight matrices to find massive values and heavy hitters in attention mechanisms, including RoPE-based models, without running inference. - Dynamic Component Ranking: Rank neurons by activation statistics to localize specialized capabilities, such as language-specific neurons in multilingual models like LLaMA-2, BLOOM, OPT, Mistral, and Phi-2. - Layer-wise Representation Comparison: Measure layer-wise distances across contrastive inputs to localize task-relevant layers, demonstrated with TruthX for editing representations to improve truthfulness. - Use Case: A researcher investigating hallucinations uses layer-wise magnitude comparison to find where truthful representations diverge, then applies TruthX editing vectors to those layers and evaluates on TruthfulQA. ## Quick Start Ask the assistant to analyze massive values in a Llama model's attention matrices and visualize which layers and heads contain the largest activations.
