mechanism-behavior-discovery

Surfaces novel falsifiable behavioral phenomena in LLMs for downstream mechanistic investigation.

75|7|Updated May 2, 2026
One-click install
npx skills add https://github.com/zjunlp/Mechanist --skill mechanism-behavior-discovery-zjunlp
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: mechanism-behavior-discovery
Source: https://github.com/zjunlp/Mechanist/tree/main/skills/mechanism-behavior-discovery
Command: npx skills add https://github.com/zjunlp/Mechanist --skill mechanism-behavior-discovery-zjunlp

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Open-ended interpretability research often stalls at the first step: deciding which model behavior is actually worth explaining. This Skill guides the discovery half of the research loop by systematically surfacing a new, non-obvious, falsifiable behavioral phenomenon in LLM or multimodal models, along with the data strategy to validate it. ## Core Features & Use Cases - Six Discovery Strategies: Transfer known phenomena into high-stakes domains (chemistry, medicine), borrow findings from human sciences, test cross-modal transfer, reuse prior CS results, probe a phenomenon's boundary conditions or causal origin, and run meta-analysis to distill laws. - Quality Bar for Candidates: Every candidate phenomenon is filtered against five criteria — real, non-obvious, specific, robust, and tractable — and sharpened into a one-sentence falsifiable claim with a data/metric plan and a plausible internal locus. - Deduplication Against Prior Work: When given a record of already-explored phenomena (established, conditional, refuted, or inconclusive), it commits to a genuinely new direction and keeps unpicked ideas as a backlog. - Use Case: A researcher asks "find something surprising about how this model behaves in clinical diagnosis." The Skill brainstorms candidates, commits to one falsifiable phenomenon, and hands it off to /mechanism-explore for mechanistic investigation. ## Quick Start Ask the agent to find a novel, testable behavioral phenomenon in your target model and domain, then hand the chosen candidate to /mechanism-explore for mechanistic analysis.

Frequently Asked Questions about mechanism-behavior-discovery

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find a novel LLM behavior worth researching?

Use one of six discovery strategies: transfer a known phenomenon into a high-stakes domain, borrow findings from psychology or neuroscience, test cross-modal transfer, reuse prior CS results, probe when or why a phenomenon arises, or distill laws via meta-analysis. Commit to one falsifiable candidate.

What makes a good behavioral phenomenon candidate for interpretability?

A candidate must be real, non-obvious, specific, robust, and tractable, meaning a plausible internal locus exists to explain it. It should be sharpened into a one-sentence falsifiable behavior with a defined dataset and metric.

When should I use behavior discovery versus mechanism exploration?

Use discovery only when the phenomenon is not already named and the task is open-ended. If the user explicitly names the behavior to explain, skip discovery and go directly to mechanism exploration of that named phenomenon.

Does this skill work with multimodal models or only text LLMs?

It is domain-general with no assumption about model family, modality, or task. Cross-modal transfer is an explicit strategy, checking whether text-domain phenomena appear in image, video, or multimodal models.

How does the skill avoid re-proposing already studied phenomena?

When given a record of prior outcomes, it excludes phenomena marked established, conditional, or not-established, since those questions are answered. Phenomena marked inconclusive remain valid retry targets, and unchosen candidates are kept as a backlog.