paperjury

Reviews research papers by independently reproducing experimental metrics and auditing claims against code outputs.

6|1|Updated May 11, 2026
One-click install
npx skills add https://github.com/yakeworld/Synthos --skill paperjury-yakeworld
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: paperjury
Source: https://github.com/yakeworld/Synthos/tree/main/skills/private/paperjury
Command: npx skills add https://github.com/yakeworld/Synthos --skill paperjury-yakeworld

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Pre-submission papers often contain experimental numbers (F1, AUC, Recall) that cannot be traced back to actual code runs, risking fabricated or mismatched claims slipping into submissions. This Skill performs a deep semantic review of a paper and audits every numeric claim against independently executed code outputs before submission. ## Core Features & Use Cases - Experimental Data Integrity Audit: Extracts all numeric claims from the paper, checks for corresponding scripts in the code directory, and labels each claim CLOSE, MISMATCH, or FABRICATED based on independent reproduction. - Multi-Round Review Pipeline: Runs a structured workflow of L1-L2 structural checks, semantic review, fixes, deep multi-reviewer deliberation, and L5-L7 compliance checks, with independent reviewers and strict token budgets (10k per round, 50k cumulative). - Use Case: Before submitting a machine learning paper claiming F1=0.92, run this Skill to re-execute the training script, archive JSON/CSV evidence, and confirm the claimed metric matches the reproduced result—or flag it as fabricated if no supporting code exists. ## Quick Start Review my paper in the papers/ directory, audit every experimental metric against the code in 03-code/, and report which claims are CLOSE, MISMATCH, or FABRICATED.

Frequently Asked Questions about paperjury

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify experimental metrics in a research paper before submission?

Extract every numeric claim (F1, AUC, Recall) from the paper, locate the corresponding scripts in the code directory, and independently re-run them. Compare claimed versus actual values and label each claim CLOSE, MISMATCH, or FABRICATED, archiving JSON and CSV evidence.

What happens when a paper metric has no supporting code?

Any metric without a corresponding script and output in the code directory is marked FABRICATED and blocked from release. The claim cannot be labeled CLOSE or approved until the author provides executable code and reproducible outputs.

Can this replace peer review for my paper?

No, it is strictly a pre-submission self-check and does not replace peer review. Issues requiring new experiments are marked OPEN and handed back to the author rather than being resolved by fabricating data.

What are the token limits for multi-round paper review?

A single review round is capped at 10k tokens and multi-round cumulative usage at 50k tokens. When the budget is exceeded, the review auto-terminates and emits a list of unresolved issues for the author.

Why must reviewers read the paper independently in each round?

Each reviewer must read the original text independently without sharing notes or conclusions across rounds. Cross-round leakage contaminates scores and undermines the independence of the deliberation.