What problem does it solve? Understanding how a transformer model builds its predictions across layers is difficult because intermediate hidden states are not directly interpretable. This Skill trains and applies tuned lenses to project intermediate residual stream states into vocabulary space, revealing how predictions evolve layer by layer. ## Core Features & Use Cases - Tuned Lens Training: Train affine translators per layer that map hidden states to vocabulary logits by minimizing KL divergence against the model's final output distribution. - Layer-wise Prediction Analysis: Extract hidden states from any HuggingFace causal LM, apply a lens at each layer, and inspect top-k token predictions, entropy evolution, and prediction trajectories. - Visualization & Comparison: Plot prediction trajectories for target tokens across layers and compare predictions at early, middle, and final layers. - Use Case: Given a prompt like "The capital of France is", trace how the model's belief about the next token shifts from generic words in early layers to "Paris" in later layers, quantifying confidence and entropy at each stage. ## Quick Start Run the tuned lens analysis on GPT-2 for the prompt "The capital of France is" and show me the top predictions at each layer.
