skills.rest
KaggleKaggleOfficial·2 Agent Skills Included

kaggle-benchmarks

Custom LLM evaluation tasks, assertions, and dataset benchmarking

Builds custom benchmark tasks to rigorously evaluate language models on any task you define. Replaces ad-hoc prompt testing with structured assertions, multimodal inputs, tool use, and multi-turn conversation checks. Runs evaluations across full datasets and multiple models, then publishes reproducible results to Kaggle leaderboards.
185|49
↗
npx skills add Kaggle/kaggle-benchmarks --all -g -y
Available:

Gives your AI agent a map of the library's architecture, key API files, and operational commands so it can write and run benchmark tasks correctly.

All Skills in This Repository (2)

Pure Emerald Level Indicators
📦 In Repo
KaggleKaggle

kaggle-benchmarks

Create and evaluate AI model benchmark tasks with structured outputs.

Official
Advanced
📦 In Repo
KaggleKaggle

kaggle-standardized-agent-exam

Register a Kaggle agent and take a 16-question standardized exam.

Official
Advanced

Frequently Asked Questions

FAQPage Schema
How to install kaggle-benchmarks?▼

Run `npx skills add Kaggle/kaggle-benchmarks --all -g -y` in your terminal to install all skills in this repository globally.

How to evaluate an LLM on a custom task?▼

Define a Python function with the @kbench.task decorator, prompt the model inside it, and validate outputs with kbench.assertions. Then call .run() or .evaluate() to execute it against one or more models.

Can I benchmark models over a whole dataset?▼

Yes. Pass a pandas DataFrame to .evaluate() to run your task across every row, with parallel workers, retries, and aggregate accuracy scores.

Does kaggle-benchmarks support images and tool use?▼

Yes. Tasks can send image, audio, and video inputs, request structured outputs via schemas, and give models tools like a built-in Python interpreter.

What is the Kaggle Standardized Agent Exam?▼

It is a skill that lets your agent register on Kaggle, take a 16-question timed exam over HTTP, and earn a public score on the leaderboard.

Related Repositories in Data & Analytics

View All in Data & Analytics→
PaddlePaddlePaddlePaddle
87.3k

PaddleOCR

Extract text, tables, and formulas from PDFs and images

Official📦 2 Skills
D4VinciD4Vinci
73.3k

Scrapling

Scrape any website and bypass anti-bot protection with AI

Community📦 1 Skills
mvanhornmvanhorn
60.8k

last30days-skill

Research any topic across Reddit, X, YouTube, and the web

Community📦 2 Skills