constructing-llm-probes

To interpret LLM activations, you extract hidden states from target layers and train linear or nonlinear probes like Logistic Regression or MLP classifiers to test what specific information is encoded in those vectors.

To locate where signals emerge across LLM layers, extract activations from each layer using output_hidden_states and evaluate trained probes across them to pinpoint the exact depth where syntactic or factual information first appears.

Yes, you can compare linear vs nonlinear encodings by training both linear classifiers like Ridge and nonlinear MLP probes on the same extracted activations to determine if the encoded information is linearly separable.

For downstream analysis of LLM activations, apply dimensionality reduction techniques like PCA and Sparse Autoencoders (SAE) to visualize and decompose the hidden state representations into interpretable features.

Yes, the workflow supports causal interventions on transformer activations, allowing you to manipulate specific hidden state dimensions and observe the resulting changes to verify hypotheses about encoded information.

Extract activations across multiple prompt styles and spans when comparing model behaviors or localizing signals, ensuring that identified information encoding is robust and not dependent on a specific input formatting.