audit-reproducibility

Audit numeric manuscript claims against R, Python, and Fortran analysis outputs.

Updated Sep 11, 2026
One-click install
npx skills add https://github.com/cameronbracken/waterology-research --skill audit-reproducibility-cameronbracken
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: audit-reproducibility
Source: https://github.com/cameronbracken/waterology-research/tree/main/skills/audit-reproducibility
Command: npx skills add https://github.com/cameronbracken/waterology-research --skill audit-reproducibility-cameronbracken

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Manuscripts often contain numbers that drift out of sync with the analysis outputs that produced them, and manual checking before submission is slow and error-prone. This Skill systematically compares every numeric claim in a paper against the actual computed results and records the verdict in a reproducibility passport. ## Core Features & Use Cases - Claim extraction and matching: Pulls point estimates, standard errors, p-values, sample sizes, and percentages from a manuscript and matches them to values in .rds, .tex, .csv, .parquet, .dat, .log, and .json outputs with a confidence score. - Tolerance-based dispositions: Applies default or per-claim tolerances and assigns PASS, FAIL, EXPLAINED, or UNMATCHED, with strict rules preventing vague explanations from downgrading failures. - Passport lifecycle tracking: Creates or refreshes a YAML reproducibility passport at quality_reports/passports/<paper-slug>.yaml, tracking STALE and UNVERIFIED states and a conformance ladder from index to verify. - Use Case: Before submitting a revised paper, run the audit against the manuscript and the results directory to confirm every reported statistic still matches the current analysis outputs, then update the passport with fresh verification timestamps. ## Quick Start Audit the numeric claims in manuscript.qmd against the outputs directory and update the reproducibility passport for this paper.

Frequently Asked Questions about audit-reproducibility

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check that numbers in my manuscript match my analysis results?▼

Run the audit with a manuscript path and one or more output files or directories. It extracts numeric claims, matches them to values in .rds, .tex, .csv, .parquet, .dat, .log, or .json outputs, and reports PASS, FAIL, EXPLAINED, or UNMATCHED per claim.

What is a reproducibility passport for a research paper?▼

A passport is a YAML record at quality_reports/passports/<paper-slug>.yaml tracking each numeric claim's provenance, tolerance, status, and verification timestamps. It follows a one-way conformance ladder of index, trace, replay, and verify levels.

What tolerances are used when comparing reported and computed values?▼

Defaults are exact match for integers and sample sizes, absolute difference under 0.01 for point estimates, under 0.05 for standard errors, same significance level for p-values, and within 0.1 percentage points for percentages. Per-claim overrides take precedence.

Does the audit rerun my analysis if outputs are missing?▼

The audit reads existing outputs and never silently reruns an analysis. If a rerun is explicitly authorized, it launches the command in the background with a preserved log and polls it, treating missing evidence as a limitation rather than a pass.

When can a FAIL be downgraded to EXPLAINED?▼

Only when the notes field names a specific alternative specification that accounts for the difference for that exact claim. Blank notes, rounding claims without quantified comparison, or general assurances do not qualify, and UNMATCHED claims can never be downgraded.