constructing-llm-probes

Name: constructing-llm-probes
Availability: InStock
Author: concordance-co

Extract LLM hidden-state activations and train linear or nonlinear probes.

4|1|Updated May 20, 2026

Search Tags:#llm #probe #transformers #interpretability #activations #pca #sae

Ready to run

npx skills add https://github.com/concordance-co/xenon --skill constructing-llm-probes

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: constructing-llm-probes
Source: https://github.com/concordance-co/xenon/tree/main/.agents/skills/constructing-llm-probes
Command: npx skills add https://github.com/concordance-co/xenon --skill constructing-llm-probes

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Probes provide a structured way to uncover what information is encoded in LLM activations, enabling researchers to assess interpretability, compare models, and localize signals across layers and spans.

Core Features & Use Cases

Activation extraction across layers with output_hidden_states
Train linear and nonlinear probes (Logistic Regression, Ridge, MLP) to test hypotheses
Analysis tools: PCA, SAE, logit lens, and causal interventions
Use cases: determine what information (syntax, sentiment, facts) is captured, compare models, and localize signals in specific layers/spans

Quick Start

Run a minimal probing workflow to extract hidden states, train a linear or nonlinear probe, and evaluate across layers.

Frequently Asked Questions about constructing-llm-probes

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I train probes on LLM hidden states to interpret activations?▼

To interpret LLM activations, you extract hidden states from target layers and train linear or nonlinear probes like Logistic Regression or MLP classifiers to test what specific information is encoded in those vectors.

What is the best way to locate where specific signals first emerge across LLM layers?▼

To locate where signals emerge across LLM layers, extract activations from each layer using output_hidden_states and evaluate trained probes across them to pinpoint the exact depth where syntactic or factual information first appears.

Can I compare linear vs nonlinear encodings in transformer models?▼

Yes, you can compare linear vs nonlinear encodings by training both linear classifiers like Ridge and nonlinear MLP probes on the same extracted activations to determine if the encoded information is linearly separable.

How do I apply PCA and SAE for downstream analysis of LLM activations?▼

For downstream analysis of LLM activations, apply dimensionality reduction techniques like PCA and Sparse Autoencoders (SAE) to visualize and decompose the hidden state representations into interpretable features.

Does this probing workflow support causal interventions on transformer activations?▼

Yes, the workflow supports causal interventions on transformer activations, allowing you to manipulate specific hidden state dimensions and observe the resulting changes to verify hypotheses about encoded information.

When do I need to extract activations across multiple prompt styles and spans?▼

Extract activations across multiple prompt styles and spans when comparing model behaviors or localizing signals, ensuring that identified information encoding is robust and not dependent on a specific input formatting.