What problem does it solve? Research papers and benchmarks often report suspiciously high accuracy caused by data leakage, incomplete baselines, or non-reproducible experiments. This Skill audits machine learning experiment directories (such as PIDD diabetes classification studies) to detect leakage, verify baseline completeness, and cross-check published claims against reproducible results. ## Core Features & Use Cases - Cross-Dataset Leakage Auditing: Runs consistent audits across datasets (PIDD, INSCAT, Diabetes 130-US) with fixed ZeroReplacer, pipeline structure, CV settings, and metrics so comparisons remain valid. - Baseline Completeness Checks: Detects gaps between recorded and claimed model counts (e.g., 27 vs 32 baselines) and names missing models such as DummyClassifier or StackingClassifier. - Literature Validation: Searches Semantic Scholar, CrossRef, and PubMed for papers reporting accuracy above 90% or F1 above 0.70, flags data leakage risk, and cross-references OpenML public experiments (PIDD ID:292) as third-party verification. - Use Case: Given a research directory claiming 32 baselines with 95% accuracy on PIDD, the audit reveals only 27 models were run, identifies the 5 missing classifiers, and shows post-leakage F1 converges near 0.70, contradicting the published claim. ## Quick Start Audit this PIDD experiment directory for data leakage, verify the baseline count matches the paper's claims, and cross-check high-accuracy papers in the literature for leakage risk.
