reproducibility-package-review

Verifies committed results can be regenerated end-to-end from committed scripts, seeds, and parameters.

1|Updated Dec 13, 2025
One-click install
npx skills add https://github.com/ZK-Theory/TDL --skill reproducibility-package-review-zk-theory
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: reproducibility-package-review
Source: https://github.com/ZK-Theory/TDL/tree/main/.agents/skills/reproducibility-package-review
Command: npx skills add https://github.com/ZK-Theory/TDL --skill reproducibility-package-review-zk-theory

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Individual results may each pass provenance review while the full result set is not reproducible: a gitignored intermediate has no committed producer, or a seed was never recorded. This Skill performs the final end-to-end check that someone could rebuild every committed result from the repository alone before a paper relies on it or it is handed off. ## Core Features & Use Cases - Producer Verification: Confirms every result file, including gitignored CSV/PKL intermediates, has its producing script committed on the branch. - Seed and Parameter Audit: Checks that every stochastic step's seed and run parameters (B, L, k, null model) are recorded in both the script and the vault. - Lint Gate Consistency: Verifies the enforced ruff version matches the pinned pre-commit revision, flagging drift and format-dirty committed files. - Use Case: Before submitting a paper, run this review to produce a regenerability table mapping each result to its producer script, recorded seeds, and intermediate presence, listing every gap with the missing artifact. ## Quick Start Review this repository's result set and confirm every committed result can be regenerated end-to-end from committed scripts plus recorded seeds and parameters.

Frequently Asked Questions about reproducibility-package-review

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify my research results are reproducible before submission?

Run an end-to-end reproducibility review that maps each result to its committed producer script, recorded seeds and parameters, and the on-disk presence of gitignored intermediates. The output is a regenerability table marking each result REPRODUCIBLE or GAP with the missing artifact listed.

Why can individual results pass provenance review while the full set is not reproducible?

Per-file provenance checks miss set-level gaps: a gitignored intermediate may have no committed producing script, or a stochastic step's seed was never recorded. The chain only closes when following recorded commands from raw inputs reproduces the committed results.

How do I check that my ruff lint gate matches the pre-commit version?

Compare the output of uv run ruff --version against the rev pinned in .pre-commit-config.yaml. When they diverge, the pinned pre-commit version governs; validate against it and flag the drift, since a format-dirty committed file signals a prior commit skipped the ruff-format hook.

Does a gitignored intermediate file mean it is missing from the repository?

No, gitignored does not mean missing. Downstream-needed gitignored files must exist at their PROJ_ROOT paths on disk, and each must have a committed producing script plus a recorded regeneration command in the Task Log or vault entry.

When should I run a reproducibility package review?

Run it at the end of the output and provenance lane, before a result set is relied on for a paper or handed off. It is the final check that someone could rebuild the results from the repository alone, complementing per-file provenance review.