exp-design

Designs claim-driven ML experiment plans with staged run order and wiki integration.

Updated Jul 15, 2026
One-click install
npx skills add https://github.com/1549171506/shizhengmubangai1 --skill exp-design-1549171506
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: exp-design
Source: https://github.com/1549171506/shizhengmubangai1/tree/main/Desktop/123/empiricalwiki/i18n/en/skills/exp-design
Command: npx skills add https://github.com/1549171506/shizhengmubangai1 --skill exp-design-1549171506

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Turning a research idea or hypothesis into a rigorous, executable experiment plan is error-prone: baselines get missed, ablations are incomplete, and success criteria stay vague. This Skill structures the entire experiment design process around claims, ensuring every experiment validates a specific claim with quantified pass/fail criteria. ## Core Features & Use Cases - Claim Scoping: Scopes claims across three dimensions (Target, Decomposition, Threats), reusing existing wiki claims or creating new ones when gaps exist. - Four Experiment Block Types: Designs baseline reproduction, validation, ablation, and robustness experiments, each with setup, metrics, seeds, and GPU-hour estimates. - Staged Run Order with Decision Gates: Orders experiments into stages (sanity check → baseline → validation → ablation → robustness) with explicit stop conditions between stages. - Optional Review LLM Pass: With --review, sends the plan to a review model that checks for missing baselines, unfair comparisons, and statistical rigor. - Use Case: Given an idea like "sparse-lora-for-edge-devices" in your research wiki, generate a complete experiment plan with wiki pages, graph edges, and a compute budget report ready for execution. ## Quick Start Ask the assistant to design an experiment plan for the idea sparse-lora-for-edge-devices with a 40 GPU-hour budget and review enabled.

Frequently Asked Questions about exp-design

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an ML experiment plan from a research idea?

Provide an idea slug from wiki/ideas/ or a free-text hypothesis. The skill scopes target, decomposition, and threat claims, then designs baseline, validation, ablation, and robustness experiment blocks with quantified success criteria and a staged run order.

What experiment types should a research plan include?

A complete plan includes baseline reproduction, core validation, per-factor ablations, and robustness tests across at least two variation dimensions. Validation and ablation experiments should specify at least three random seeds for statistical reliability.

How does the compute budget limit affect experiment design?

Pass --budget with a GPU-hour cap to constrain the plan. If the estimate exceeds the budget, the robustness stage scope is reduced first, and the actual allocation is noted in the experiment plan report.

Can the experiment plan be reviewed automatically before running?

Yes, the --review flag sends the full plan to a Review LLM that checks for missing baselines, unfair comparisons, insufficient ablations, and statistical concerns. The plan is revised based on its feedback before being written to the wiki.

What happens if the target claim does not exist in the wiki?

The skill auto-creates a new claim page with status proposed and confidence 0.3, then flags it in the report. Every experiment must link to a claim, so the target_claim field is never left empty.