experiment-audit

Audit machine learning experiment results for fraudulent reporting patterns.

2|Updated Aug 12, 2025
One-click install
npx skills add https://github.com/goupup-ai/miccai25 --skill experiment-audit-goupup-ai
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: experiment-audit
Source: https://github.com/goupup-ai/miccai25/tree/main/ARIS/skills/experiment-audit
Command: npx skills add https://github.com/goupup-ai/miccai25 --skill experiment-audit-goupup-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

LLM agents running machine learning experiments can inadvertently produce fraudulent results, including fake ground truth derived from model outputs, manipulated score normalization, phantom unrun experiments, and overstated evaluation scope. This Skill adds integrity constraints to catch these failure modes before results are used to support research claims.

Core Features & Use Cases

  • Cross-model independent review: Uses an external reviewer backend to evaluate experiment artifacts without executor bias, ensuring judgment separation.
  • Fraud pattern detection: Checks for fake ground truth, score normalization fraud, phantom results, dead code, and scope mismatches between claims and actual testing.
  • Pipeline integration: Automatically tags research claims with integrity status (PASS/WARN/FAIL) without blocking workflow progression, and generates structured audit reports for machine and human consumption.
  • Use Case: A researcher preparing a conference paper can run this Skill after completing experiments to audit result validity before writing claims, avoiding retractions or credibility issues from accidental fraud.

Quick Start

Use the experiment-audit skill to audit the integrity of all experiment results in your current project before finalizing research paper claims.

Frequently Asked Questions about experiment-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect fake ground truth in machine learning experiment results?

Machine learning experiment validation involves verifying the integrity of evaluation scripts, result files, and ground truth provenance to detect fraudulent reporting patterns. It ensures claims align with actual testing scope before publication.

What is score normalization manipulation in ML research claims?

Score normalization manipulation is a fraudulent reporting pattern where evaluation metrics are artificially adjusted. Independent cross-model review detects this by applying integrity constraints to result files and auditing experiment artifacts.

How do I audit ML experiment integrity before finalizing research paper claims?

To audit ML experiment integrity before finalizing research paper claims, run an independent review of your experiment artifacts. This evaluates evaluation scripts, result files, and ground truth provenance to tag research claims with PASS, WARN, or FAIL integrity status.

Can I integrate experiment result fraud detection into my existing research workflow pipeline?

Yes, you can integrate experiment result fraud detection into your research workflow pipeline. It automatically tags research claims with integrity status without blocking workflow progression, generating structured audit reports for both machine and human consumption.

What are common fraudulent reporting patterns in machine learning experiments?

Common fraudulent reporting patterns in machine learning experiments include fake ground truth, score normalization fraud, phantom unrun experiments, dead code, and overstated evaluation scope. These failure modes occur when LLM agents inadvertently produce fraudulent results during execution.