What problem does it solve? When reporting classification results (e.g., on the PIDD diabetes dataset), researchers need to position their method against public OpenML baselines without misattributing F1 gaps to preprocessing or misusing Weka-specific methods as core arguments. ## Core Features & Use Cases - Benchmark Fetching: Pulls run lists and per-run evaluation metrics from the OpenML REST API with batching (limit=500) and timeout controls (--max-time 30). - Data Hygiene: Converts string-typed evaluation values to floats, filters invalid runs (accuracy/f1 = 0), and recomputes means independently. - Gap Attribution: Ranks F1 gap causes (model implementation differences > no SMOTE > no standardization > feature selection portability) and flags Weka-specific methods as not portable to sklearn. - Use Case: Given a CatBoost result of F1=0.7759 on PIDD, produce a comparison table against 50+ OpenML runs (WEKA RF F1=0.7648, best 0.8026) with the method's rank and a defensible attribution of the remaining gap. ## Quick Start Ask the agent to fetch OpenML benchmark runs for the PIDD dataset and compare my CatBoost F1 score against the public baselines with a ranked attribution of the gap.
