What problem does it solve? Understanding which neurons control language-specific behavior in multilingual LLMs requires tedious manual analysis of activations across layers. This Skill provides scripts to load pre-identified language neurons, create deactivation masks, and measure the impact on perplexity and generation. ## Core Features & Use Cases - Neuron Loading & Analysis: Load pre-identified language-specific neurons from .neuron.pth files for LLaMA-2, BLOOM, OPT, Mistral, and Phi-2, with per-layer distribution statistics and cross-language overlap comparison. - Deactivation Mask Generation: Create binary activation masks for MLP or attention neurons at configurable deactivation ratios, including progressive deactivation experiments. - Perplexity Impact Measurement: Compute perplexity with and without neuron deactivation across languages (en, zh, fr, es, vi, id, ja) and visualize cross-lingual impact matrices. - Use Case: Deactivate Chinese-specific neurons in LLaMA-2-7B and measure how perplexity changes for Chinese versus other languages to verify neuron language specificity. ## Quick Start Analyze the language-specific neurons in LLaMA-2-7B and create a deactivation mask for Chinese neurons, then measure the resulting perplexity change.
