experiment-scaffold

Scaffold an approval-gated experimental framework with baselines and walking-skeleton validation.

Updated Aug 23, 2026
One-click install
npx skills add https://github.com/UnaryLab/ai-for-research --skill experiment-scaffold-unarylab
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: experiment-scaffold
Source: https://github.com/UnaryLab/ai-for-research/tree/main/skills/experiment-scaffold
Command: npx skills add https://github.com/UnaryLab/ai-for-research --skill experiment-scaffold-unarylab

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It prevents wasting GPU-months by forcing a plan-first, approval-gated experimental framework that can falsify your research claim with correct baselines, metrics, datasets/workloads, and statistics.

Core Features & Use Cases

  • Plan-first experiment & framework contract: Produces a complete experiment + harness plan (decision rule, variables, baselines, datasets/workloads, ablations, statistical plan, and result→artifact map) before any framework code is created.
  • Reference-aligned harness architecture: Specifies the end-to-end code organization (declarative configs, module interfaces, reproducibility, experiment tracking, and orchestration) plus a minimal walking-skeleton milestone.
  • Walking-skeleton validation workflow: Builds an end-to-end pipeline on toy scale, validates determinism and metric logging, and only then proceeds to real method or real-scale runs.

Quick Start

Ask the skill to produce an approval-ready experiment-plan for your hypothesis, including baselines, datasets/workloads, ablations, and the exact decision rule, and wait for sign-off before it scaffolds any project files.

Frequently Asked Questions about experiment-scaffold

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design a falsifiable experiment harness before writing machine learning code?

To design a falsifiable experiment harness before coding, you create a plan-first contract defining baselines, ablations, metrics, datasets, and a decision rule. This enforces end-to-end validation on toy scale and artifact traceability before real-scale runs.

What is a walking skeleton in experiment tracking and research reproducibility?

A walking skeleton in research reproducibility is a minimal end-to-end pipeline built on toy scale data. It validates determinism and metric logging before proceeding to real-scale method runs, ensuring experiment tracking and artifact traceability function correctly.

How to structure an ablation study and baseline comparison for computer architecture research?

Structuring an ablation study and baseline comparison requires defining variables, datasets, workloads, and a statistical plan. A falsifiable framework contract maps results to artifacts, ensuring reference-aligned harness architecture for simulators and scientific computing experiments.

Does this experiment scaffolding approach work for simulator and workload suite benchmarking?

Yes, experiment scaffolding applies directly to computer-architecture research using simulators and workload suites. It defines the harness architecture, walking skeleton, and validation workflow required to ensure reproducibility and correct benchmarking before real-scale runs.

Why do I need an approval gate before scaffolding experiment framework files?

An approval gate prevents wasting GPU-months by requiring a complete experiment plan, including baselines, ablations, and exact decision rules, before creating any framework code. This ensures falsifiable evidence generation and artifact traceability for your research claim.

What are the limitations of using a plan-first experimental framework for machine learning research?

A plan-first experimental framework requires waiting for sign-off on a complete harness plan before scaffolding any project files. This strict approval gate and walking-skeleton validation on toy scale may delay rapid prototyping or exploratory machine learning iterations.