experiment-design

Generate seeds, splits, power estimates, stopping rules, and reporting templates for reproducible experiments in Python with NumPy, pandas, scikit-learn, and statsmodels.

242|64|Updated Jan 2, 2026
One-click install
npx skills add https://github.com/Yeachan-Heo/My-Jogyo --skill experiment-design-yeachan-heo
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: experiment-design
Source: https://github.com/Yeachan-Heo/My-Jogyo/tree/main/skills/experiment-design
Command: npx skills add https://github.com/Yeachan-Heo/My-Jogyo --skill experiment-design-yeachan-heo

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill standardizes experimental design to ensure reproducibility.

Core Features & Use Cases

  • Reproducibility setup: Seeds, environment recording, and versioning to ensure experiments can be replicated.
  • Experimental controls: Train/test splits, cross-validation, and power analyses.
  • Pre-registration & reporting: Templates to pre-register analysis plans and clearly label confirmatory vs exploratory findings.

Quick Start

  • Load this skill to structure an experiment from goal to report.
  • Provide your research objective, dataset context, and analysis plan; the skill will generate seeds, environment checks, data splits, power estimates, stopping rules, preregistration, and a reporting template.
  • Run the templates and adapt them to your project for immediate reproducible results.

Frequently Asked Questions about experiment-design

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up reproducible experiments in Python with proper seeds and environment recording?

Reproducible experiments require fixed random seeds, environment recording, and versioning. This skill generates those setup checks alongside data splits and cross-validation templates to ensure your Python analysis can be fully replicated later.

What is pre-registration in data science and how do I create a template for it?

Pre-registration records your analysis plan before running experiments to distinguish confirmatory from exploratory findings. This skill provides pre-registration templates that clearly label results, preventing post-hoc bias in your reporting.

How do I perform a power analysis for a machine learning experiment using statsmodels?

Power analysis estimates the sample size needed to detect an effect reliably. This skill generates power estimates using statsmodels and numpy, integrating them into your experimental design to establish statistically valid stopping rules.

Can I use this skill to standardize train/test splits and cross-validation for scikit-learn models?

Yes, this skill standardizes experimental controls by generating train/test splits and cross-validation configurations for scikit-learn. It structures these controls within a reproducible framework to maintain consistency across data science experiments.

Do I need pandas and numpy installed to structure my experimental design from goal to report?

Yes, this skill requires a Python environment with numpy, pandas, scikit-learn, and statsmodels installed. These dependencies power the data splitting, power analysis, and environment recording needed to generate reproducible experiment templates.

Why does my data science experiment lack reproducibility even with fixed random seeds?

Reproducibility fails without environment recording and versioning alongside fixed seeds. This skill addresses this by standardizing the entire experimental design, adding data splitting rules, preregistration, and reporting templates to ensure full replicability.