paper-claim-audit

Audit research paper claims against raw experimental evidence files.

Updated Apr 21, 2026
One-click install
npx skills add https://github.com/Shallow-W/llm-wiki --skill paper-claim-audit-shallow-w
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: paper-claim-audit
Source: https://github.com/Shallow-W/llm-wiki/tree/main/.claude/skills/paper-claim-audit
Command: npx skills add https://github.com/Shallow-W/llm-wiki --skill paper-claim-audit-shallow-w

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Prevents confirmation bias and accidental (or intentional) misreporting by checking that every quantitative claim in a paper exactly matches the corresponding raw result files.

Core Features & Use Cases

  • Zero-context paper-to-evidence audit: Reviews only paper sources and raw result/config artifacts, without executor narratives or prior audit context.
  • Claim-level reconciliation: Extracts every number/percentage/comparison/scope statement and traces it to the exact evidence value.
  • Rigorous mismatch taxonomy: Detects exact mismatches, rounding drift, config mismatches, aggregation/run-count errors, delta arithmetic errors, caption-to-figure inconsistencies, and scope overclaims.
  • Machine-readable assurance output: Always emits paper/PAPER_CLAIM_AUDIT.json plus a human report at paper/PAPER_CLAIM_AUDIT.md for downstream automation.

Quick Start

Ask the system to run paper-claim-audit for your current paper directory and associated raw results before submission or after paper writing changes.

Frequently Asked Questions about paper-claim-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify paper claims against raw experiment data before submission?

To verify paper claims before submission, run a zero-context audit that traces every numeric and comparative statement to exact raw result files, outputting PASS/WARN/FAIL statuses and a PAPER_CLAIM_AUDIT.json artifact.

What is a zero-context paper-to-evidence audit?

A zero-context audit reviews only paper sources and raw result/config artifacts without executor narratives or prior context, ensuring quantitative claim reconciliation remains free from confirmation bias.

How do I check for rounding drift and delta arithmetic errors in research papers?

Check for rounding drift and delta arithmetic errors by applying a rigorous mismatch taxonomy that detects exact mismatches, config mismatches, aggregation errors, and caption-to-figure inconsistencies during claim extraction.

Can I audit scope overclaims and aggregation errors in my paper directory?

Yes, you can audit scope overclaims and aggregation errors by running the audit on your current paper directory and associated raw results, which checks that every quantitative claim matches the corresponding raw result files.

What's the best way to generate machine-readable assurance outputs for paper verification?

The best way to generate machine-readable assurance outputs is to run an audit that always emits a paper/PAPER_CLAIM_AUDIT.json file plus a human-readable PAPER_CLAIM_AUDIT.md report for downstream automation.

When should I not use an automated paper claim audit?

You should not use it when executor narratives or prior audit context are the primary sources of truth, because the zero-context protocol strictly reviews only paper sources and raw result/config artifacts.