ml-paper-research

Query Papers with Code leaderboards and arXiv for the task, then collect 3-8 high-signal primary papers. The workflow reads each paper structurally and produces a SOTA summary with caveats and a comparison table of as-reported results.

Extract each paper's setup, datasets, baselines, and reported metrics, then build a comparison table. Only compare numbers from comparable setups; when setups differ, the comparison is explicitly flagged as not apples-to-apples.

The workflow uses arXiv for preprints, Papers with Code for leaderboards and code links, OpenReview for peer reviews, Semantic Scholar for citation graphs, Google Scholar for broad search, and official conference proceedings for camera-ready versions.

No. The skill requires fetching and reading primary sources with the web tool rather than relying on memory. Any claim that cannot be verified from a fetched source is labeled [unverified] with a note on what would confirm it.

Define the specific claim to reproduce, then document code, weights, and dataset availability, environment and hyperparameters, under-specified details with resolution strategies, and exact baselines and metrics. The plan ends with concrete implementation tasks.

Reported metrics often come from different datasets, compute budgets, and evaluation protocols, so cross-paper numbers are not always comparable. The workflow mitigates this by attributing every metric to its exact setup and flagging invalid comparisons rather than merging them.