add-model

Integrates new tabular ML models into the TabArena benchmark as AutoGluon wrappers.

313|74|Updated May 15, 2023
One-click install
npx skills add https://github.com/autogluon/tabarena --skill add-model-autogluon
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: add-model
Source: https://github.com/autogluon/tabarena/tree/main/.claude/skills/add-model
Command: npx skills add https://github.com/autogluon/tabarena --skill add-model-autogluon

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Adding a new machine learning model to the TabArena benchmark requires creating and editing many coordinated files — an AutoGluon wrapper, an HPO search-space generator, model metadata, pyproject extras, and leaderboard family classification — and mistakes in any of them silently break discovery, testing, or leaderboard display. This Skill walks through the entire integration with codebase-specific conventions and templates. ## Core Features & Use Cases - Complete file scaffolding: Creates the model folder under packages/tabarena/src/tabarena/models/<ModelKey>/ with __init__.py, model.py, hpo.py, and info.py, auto-discovered by the model registry with no per-model test file needed. - Base-class and pattern guidance: Selects AbstractTorchModel vs AbstractModel based on framework (torch, JAX/Flax, sklearn-like), enforces the _fit contract (validation split, time limit, CPU/GPU budgets), and covers warm-up declarations and shared pretrained weights for foundation models. - Registry and packaging edits: Updates the lazy-import map, pyproject.toml extras (benchmark/extended/all unions), and the leaderboard model-family mapping, with a drift checker to keep extras in sync. - Use Case: You want to benchmark a new foundation model like TabPFN-2.6 in TabArena. Provide the model name, pip package, and docs URL, and the Skill generates the wrapper, search space, metadata, and packaging edits following in-repo reference implementations. ## Quick Start Add the TabStar model to the TabArena benchmark using the tabstar pip package and its GitHub documentation URL.

Frequently Asked Questions about add-model

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new model to the TabArena benchmark?

Create a folder under packages/tabarena/src/tabarena/models/<ModelKey>/ containing model.py (the AutoGluon wrapper), hpo.py (search-space generator), info.py (ModelInfo and MethodMetadata), and an __init__.py. The registry auto-discovers it, and test_all_models.py fit-tests it automatically.

Should my TabArena model wrapper use AbstractTorchModel or AbstractModel?

Use AbstractTorchModel only for torch-based models, since it provides get_device and _set_device device management. Non-torch GPU models like JAX/Flax libraries and CPU sklearn-like models inherit AbstractModel and add GPU resource attributes without implementing device methods.

Do I need to write a test file for a new TabArena model?

No per-model test file is needed. tests/tabarena/models/test_all_models.py is parametrized over the model registry and fits the new model automatically once its info.py is discoverable, skipping on missing optional dependencies or unavailable GPUs.

How do foundation models share pretrained weights in TabArena?

Declare a shared_weights ClassVar naming the library's loader call and the inputs that decide which network is built. AutoGluon's AbstractTorchModel memoizes the network per process so bagged folds and refits reuse one build instead of reloading per fold.

Why does my new model show as Other on the TabArena leaderboard?

The model's config_type was not added to get_model_family in website_format.py. Add the uppercased config_type to the correct family list (Tree-based, Foundation Model, Neural Network) so prefix matching assigns the proper leaderboard Type.

When should I use the add-system skill instead of add-model?

Use add-system when the integration is a self-contained ML system that does its own preprocessing, validation, HPO, or ensembling, such as AutoML frameworks, multi-model stacks, or agent pipelines. Single models fit by TabArena's shared harness use add-model.