skills.rest
Shallow-W
Shallow-WCOMMUNITY
Explore all skills by this author →
GitHub

experiment-audit

Audit completed experiment workflows for integrity issues and produce PASS/WARN/FAIL evidence.

Updated Apr 21, 2026
Search Tags:#research reporting#fraud detection#cross-model review#experiment audit#metrics validation#ground truth provenance#evaluation integrity
One-click install
npx skills add https://github.com/Shallow-W/llm-wiki --skill experiment-audit-shallow-w
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: experiment-audit
Source: https://github.com/Shallow-W/llm-wiki/tree/main/.claude/skills/experiment-audit
Command: npx skills add https://github.com/Shallow-W/llm-wiki --skill experiment-audit-shallow-w

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Prevents misleading or fraudulent experiment claims by auditing how ground truth, metrics, and results were produced and reported.

Core Features & Use Cases

  • Cross-model integrity review: Separates the executor’s artifact collection from a different-model auditor that reads and judges the evidence.
  • Fraud/failure-mode checks: Flags fake ground truth, score normalization to self-derived maxima, phantom/mismatched result reporting, dead/unused metric code, and scope overreach.
  • Non-blocking pipeline integration: Produces EXPERIMENT_AUDIT.md and EXPERIMENT_AUDIT.json and returns PASS/WARN/FAIL without halting downstream work.

Quick Start

Use the experiment-audit skill to audit completed experiment outputs before you write results or paper claims, by giving it the experiment directory or results path.

Frequently Asked Questions about experiment-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit experiment workflows for fake ground truth or mismatched result reporting?▼

You can audit experiment workflows by using a two-model review process where an executor collects file paths and a separate reviewer model reads the artifacts read-only to return structured PASS/WARN/FAIL evidence for integrity issues.

What is experiment integrity verification and when do I need to check evaluation scope?▼

Experiment integrity verification checks how ground truth, metrics, and results were produced to prevent misleading claims, needed after experiments complete to verify evidence before writing paper or narrative results.

How do I detect dead metric code or self-referential score normalization in research reporting?▼

Detect dead metric code and self-referential score normalization by applying a cross-model audit to completed experiment outputs, scanning evaluation scripts, result artifacts, configs, and declared metrics for failure-mode evidence.

Can I integrate experiment audit checks into a non-blocking pipeline?▼

Yes, you can integrate experiment audit checks non-blocking by producing EXPERIMENT_AUDIT.md and EXPERIMENT_AUDIT.json outputs that return PASS/WARN/FAIL statuses without halting downstream pipeline work.

Does cross-model review require separate models for artifact collection and evidence judging?▼

Yes, cross-model review requires a two-model process where the executor model only collects file paths and a different reviewer model reads the listed files read-only to judge evidence and return structured audit results.

Related Skills

Explore More Education & Research Skills→
zhuyingqinzhuyingqin
1

experiment-audit

Audit experiment integrity and generate structured JSON findings.

Community
Advanced
duypham2801duypham2801

experiment-audit

Audit experimental artifacts for untrustworthy reporting and generate PASS/WARN/FAIL findings.

Community
Advanced
ancolemanancoleman
503

evaluating-llms

Evaluate LLM systems with automated metrics, LLM-as-judge, and benchmarks.

Community
Advanced
sencersoylusencersoylu

ml-evaluation

Compute and summarize machine learning evaluation metrics for classifiers and regressors.

Community
Intermediate