replication

Plans and executes empirical replications of papers, claims, and benchmarks with claim ledgers.

Updated Sep 11, 2026
One-click install
npx skills add https://github.com/cameronbracken/waterology-research --skill replication-cameronbracken
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: replication
Source: https://github.com/cameronbracken/waterology-research/tree/main/skills/replication
Command: npx skills add https://github.com/cameronbracken/waterology-research --skill replication-cameronbracken

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Verifying whether a published paper's results actually hold up requires careful planning, environment control, and disciplined evidence tracking. This Skill structures the entire replication workflow so claims are tested systematically rather than ad hoc, with every result tied to its script, seed, and command. ## Core Features & Use Cases - Claim Ledger Tracking: Builds a ledger linking each paper claim to its dataset, method, hyperparameters, compute, and observed result, with assessments of aligned, partially aligned, inconclusive, or not attempted. - Environment-Gated Execution: Requires an explicit environment choice (local directory, Git branch, Pixi environment, remote SSH/Slurm, or plan-only) before any packages are installed or experiments run. - Waterology CLI Integration: Uses waterology reproduce, workflow register, and deliverable export commands to restore environments, execute through TORC, and collect run evidence. - Use Case: A researcher wants to verify the headline benchmark table from a recent machine learning paper. The Skill extracts the recipe, checks dataset availability, plans downscaled runs, executes claim by claim after environment confirmation, and produces a report with a reproducible evidence trail. ## Quick Start Ask the AI to plan a replication of a specific paper's main result, starting with a claim ledger and an explicit environment choice before any execution.

Frequently Asked Questions about replication

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I replicate results from a research paper?▼

Start by building a claim ledger from the paper's headline table or figure, linking each claim to its dataset, method, hyperparameters, and compute. Then choose an explicit execution environment and work claim by claim, recording seeds, commands, and raw outputs.

How to reproduce a computational experiment with Waterology?▼

Use waterology reproduce final with a target directory and compute profile to restore the source, environment, and inputs, then execute through TORC. For new replications, register the task with waterology workflow register replicate before running.

Can I run a replication on remote GPU or Slurm clusters?▼

Yes, remote SSH or Slurm is one of the explicit environment options for GPU or long-running experiments. The Skill requires you to confirm this environment choice before any training or execution begins.

What happens if my replication results diverge from the paper?▼

Divergent claims are assessed as partially aligned or inconclusive under this setup, never as proof the paper is wrong. You quantify what your run observed and its uncertainty, and state every downscaling or substitution made.

When should I use plan-only mode for a replication?▼

Plan-only mode is appropriate when you need to scope the code, data, metrics, and checks before committing compute resources. It separates verified facts, inferences, and missing information without executing any experiments.