skills.rest
maxwell2732
maxwell2732COMMUNITY
Explore all skills by this author →
Website𝕏@ChenZhu_CAUGitHub

check-reproducibility

Re-run a fresh-clone Stata pipeline and diff outputs to detect drift.

220|358|Updated Apr 29, 2026
Search Tags:#research automation#reproducibility#version upgrade#stata#pipeline verification#diffing outputs#output regression
One-click install
npx skills add https://github.com/maxwell2732/codex-stata-for-economists --skill check-reproducibility
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: check-reproducibility
Source: https://github.com/maxwell2732/codex-stata-for-economists/tree/main/.claude/skills/check-reproducibility
Command: npx skills add https://github.com/maxwell2732/codex-stata-for-economists --skill check-reproducibility

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill prevents silent changes in results by verifying that a full fresh-clone run reproduces the committed pipeline outputs without drift.

Core Features & Use Cases

  • Fresh-clone simulation: Re-runs the entire pipeline in a cleaned environment (while preserving data/raw/) to emulate real reproducibility conditions.
  • Output diffing with categorization: Diffs the newly generated output/ against a snapshot and classifies drift into FAIL/WARN/PASS, with special handling for numerical tables versus visual artifacts.
  • Failure-safe reporting: Surfaces whether the pipeline itself failed, which files differ, what category of drift occurred, and likely suspects (e.g., seed randomness, package/version drift, missing committed inputs).

Use case example: After upgrading Stata or touching model code, run this Skill to confirm that tables (especially .csv) match exactly and figures did not unexpectedly change.

Quick Start

In your local repo, run the Skill command that triggers the reproducibility check and outputs a PASS/WARN/FAIL verdict with a diff summary.

Frequently Asked Questions about check-reproducibility

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check Stata pipeline reproducibility before a paper submission?▼

Stata pipeline reproducibility is verified by running a fresh-clone simulation that reruns repository scripts while preserving raw data, then diffs regenerated output against committed snapshots to detect drift before paper submission.

How does output regression detection work for numerical tables and visual artifacts?▼

Output regression detection works by diffing newly generated output files against committed snapshots, applying special handling to categorize drift in numerical CSV tables separately from visual PNG or PDF artifacts.

Why does my Stata pipeline output drift after a version upgrade?▼

Stata pipeline output drift after a version upgrade often stems from package or version drift, seed randomness, or missing committed inputs, which a fresh-clone reproduction identifies by classifying differing files into FAIL, WARN, or PASS categories.

Can I verify result stability without losing my raw data during a clean git snapshot?▼

Result stability verification during a clean git snapshot preserves the raw data directory while emulating real reproducibility conditions, rerunning the codex-driven Stata pipeline in a cleaned environment to ensure outputs match committed results.

What is the best way to detect silent changes in CSV tables and PDF figures before a release?▼

The best way to detect silent changes in CSV tables and PDF figures before a release is a failure-safe reporting mechanism that diffs regenerated output against committed snapshots, surfacing which files differ and categorizing the drift type.

Related Skills

Explore More Education & Research Skills→
wangyouanwangyouan

stata-regression-workflow

Automate Stata regression workflows with log inspection and CSV/XLSX result reporting.

Community
Intermediate
dhofheinzdhofheinz
9

Commit Error Handling Skill

Diagnoses Git commit errors and repository state, outputting standardized JSON with remediation steps.

Community
Intermediate
KYRIE66nbKYRIE66nb

paper-figure

Generate publication-quality figures and tables from experiment data.

Community
Intermediate
rukikotoorukikotoo
6

stata

Execute Stata do-files via Bash and retrieve output logs.

Community
Advanced