autogluon

You can add new tabular ML models or full AutoML systems to the TabArena benchmark, run registered models from local smoke fits to cluster-scale evaluations, upload and register benchmark results to R2, regenerate leaderboard website artifacts, and adapt TabArena/bencheval as the task-model-metric layer for a domain-specific external benchmark.

They target TabArena maintainers and ML researchers who integrate models or AutoML frameworks (AutoGluon, LightAutoML, FLAML, auto-sklearn), benchmark them on the shared tuning protocol, and publish results to the leaderboard. They also serve external teams building domain benchmarks (e.g. genomics, spectroscopy) on top of TabArena's model zoo and bencheval's leaderboard math.

Invoke the benchmark-model skill with the model's registry name. It scaffolds a run script with smoke, setup, and eval subcommands, resolves the run venv, installs the model's pip extra, smoke-fits locally (CPU or GPU), then either launches sbatch jobs and monitors progress end-to-end or hands launch and monitoring to the maintainer before evaluating and reporting the leaderboard.

The upload-method skill requires a Python environment with the tabarena[benchmark] extra installed and valid R2 credentials for hosting artifacts. It inspects the run's output directory, fixes info.py, processes results.pkl files, performs a dry-run then real R2 upload, and registers the method in methods.py, handing off steps via command sheet when the environment cannot run them.

Yes. The adapt-tabarena skill guides consuming TabArena and bencheval from an external repository: converting datasets into UserTasks, applying the repeated k-fold group-aware splitting protocol, registering domain models beside TabArena's registry, layering preprocessing without forking, handling imputation and Elo anchoring, hosting results, and avoiding the git-dependency PyPI trap.