skills.rest
zjunlp
zjunlpOFFICIAL
Explore all skills by this author →
Website𝕏@ChenHuajunGitHub

dissecting-factual-predictions

Analyze factual recall in GPT-2 and GPT-J models using attention knockout and hidden state patching.

75|7|Updated May 2, 2026
Search Tags:#transformers#mechanistic interpretability#causal analysis#gpt-2#attention knockout#hidden state patching#factual recall
One-click install
npx skills add https://github.com/zjunlp/Mechanist --skill dissecting-factual-predictions-zjunlp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: dissecting-factual-predictions
Source: https://github.com/zjunlp/Mechanist/tree/main/skills/mechanism-skills/causal-attribution/ablation
Command: npx skills add https://github.com/zjunlp/Mechanist --skill dissecting-factual-predictions-zjunlp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, transformers, numpy, matplotlib, and includes scripts (resource) and references (resource) components.

What problem does it solve? Understanding where and how large language models store and retrieve factual knowledge is difficult without direct access to internal activations. This Skill provides ready-to-run intervention experiments that reveal which layers and sublayers drive factual predictions. ## Core Features & Use Cases - Attention and MLP Knockout: Zero out specific sublayers or attention heads to measure their causal contribution to a factual prediction. - Hidden State Patching: Patch activations from a source prompt into a target prompt at any layer to trace information flow. - Vocabulary Projection: Project hidden states and MLP outputs through the LM head to interpret intermediate representations as tokens. - Use Case: Given the prompt "The capital of France is", run a layer-wise knockout sweep to identify which attention and MLP layers are critical for predicting "Paris", then visualize per-layer importance scores. ## Quick Start Run the intervention experiments script on the prompt "The capital of France is" to identify the critical layers behind the model's factual prediction.

Frequently Asked Questions about dissecting-factual-predictions

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I perform attention knockout on GPT-2 in PyTorch?▼

Register a forward hook on model.transformer.h[layer_idx].attn that returns zeroed outputs, then compare predictions before and after the knockout. The included scripts implement this via the InterventionAnalyzer and ModelAnalyzer classes.

How to patch hidden states between prompts in a language model?▼

Run the model on a source prompt with output_hidden_states=True, extract the hidden state at a chosen layer and position, then inject it into the target forward pass using a forward hook. The patch_hidden_states method automates this workflow.

Does this analysis work with GPT-J or only GPT-2?▼

The scripts default to GPT-2 but support GPT-J with module name adjustments, since GPT-J uses fc_in and fc_out for MLP layers instead of c_fc and c_proj. GPT-J-6B requires an A100 GPU with 40GB VRAM.

What GPU do I need for GPT-2 XL intervention experiments?▼

GPT-2 base and medium run on any modern GPU with 8GB VRAM, GPT-2 large needs 16GB, and GPT-2 XL requires a V100 with 16GB or better. CPU execution works for small models but is slow.

Why does my forward hook not capture attention weights in GPT-2?▼

GPT-2 attention modules return tuples where weights appear only when output_attentions=True is passed to the forward call. Ensure you handle tuple outputs and request attention weights explicitly in the model call.

Related Skills

Explore More Education & Research Skills→
langwatchlangwatch
3.5k

generate-rag-dataset

Generate synthetic Q&A evaluation datasets from a RAG knowledge base for LangWatch experiments.

Official
Intermediate
MesferAliMesferAli

pyvene-interventions

Perform causal interventions on PyTorch models with a declarative dict-based framework.

Community
Advanced
0h-n00h-n0

research.attention.probe

Sets up TransformerLens and nnsight intervention experiments for mechanistic interpretability analysis of LLMs.

Community
Advanced
Orchestra-ResearchOrchestra-Research
11.5k

sparse-autoencoder-training

Train and analyze Sparse Autoencoders to decompose neural network activations into interpretable features.

Official
Advanced