experiment-plan

Converts a refined research proposal into a claim-driven experiment roadmap with run order and budgets.

75|7|Updated May 2, 2026
One-click install
npx skills add https://github.com/zjunlp/Mechanist --skill experiment-plan-zjunlp
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: experiment-plan
Source: https://github.com/zjunlp/Mechanist/tree/main/skills/experiment-plan
Command: npx skills add https://github.com/zjunlp/Mechanist --skill experiment-plan-zjunlp

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? It turns a stable research proposal or method idea into a concrete, paper-oriented experiment plan, so you know exactly which experiments to run, in what order, and what evidence each one must produce to defend your claims. ## Core Features & Use Cases - Claim Mapping: Freezes primary and supporting claims, defines anti-claims to rule out, and links every experiment block to the claim it defends. - Experiment Block Specification: Fully specifies datasets, provenance, baselines, ablations, metrics, success criteria, and failure interpretations for each block. - Run Order and Budgeting: Produces milestone-based execution stages with compute estimates, decision gates, and risk mitigations, separating must-run from nice-to-have runs. - Use Case: After refining a novel LLM interpretability method, use this Skill to generate EXPERIMENT_PLAN.md and EXPERIMENT_TRACKER.md files that define the main table, ablation matrix, and the first three runs to launch. ## Quick Start Ask the assistant to create a detailed experiment plan for your refined research proposal, including ablations, baselines, run order, and compute budget.

Frequently Asked Questions about experiment-plan

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create an experiment plan for a machine learning paper?

Start from a refined proposal, freeze the primary and supporting claims, then design experiment blocks that each defend one claim. Specify datasets, baselines, metrics, and success criteria per block, and order runs from sanity checks through decisive ablations.

What should an ablation matrix include for an LLM research paper?

An ablation matrix should isolate the dominant contribution, include a simplicity check against an overbuilt variant, and a frontier necessity check against the strongest simpler alternative. Prefer a few strong baseline families over many weak ones.

When should I use experiment planning versus running experiments directly?

Plan first when the method is stable but the validation story is unclear, since planning defines which runs actually defend the paper claims. Skip straight to execution only for trivial sanity checks or well-established evaluation protocols.

How many baselines and seeds should an experiment plan include?

The plan defaults to at most three baseline families and three seeds when stochastic variance matters and budget allows. Stronger modern baselines are preferred over padded lists of weak comparisons.

Can the experiment plan handle limited GPU compute budgets?

Yes, the plan estimates GPU-hours per milestone and separates must-run from nice-to-have experiments. Dataset subsets are only planned when a scientific reason or genuine budget constraint applies, not merely to save cost.