kaggle-benchmarks

Name: kaggle-benchmarks
Availability: InStock
Author: Kaggle

Create and evaluate AI model benchmark tasks with structured outputs.

185|49|Updated Dec 1, 2025

Search Tags:#benchmarking #AI evaluation #task definition #large language models #model assessment

Ready to run

npx skills add https://github.com/Kaggle/kaggle-benchmarks --skill kaggle-benchmarks

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: kaggle-benchmarks
Source: https://github.com/Kaggle/kaggle-benchmarks/tree/main/skills/kaggle-benchmarks
Command: npx skills add https://github.com/Kaggle/kaggle-benchmarks --skill kaggle-benchmarks

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires kaggle_benchmarks, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill empowers AI coding agents to write and evaluate benchmark tasks for AI models, ensuring reproducibility and robust evaluation.

Core Features & Use Cases

Benchmark Task Creation: Define custom tasks using decorators and structured outputs.
Model Evaluation: Interact with various large language models and assert correctness of outputs.
Use Case: Design a benchmark to test a model's ability to answer mathematical questions, ensuring it provides accurate and relevant responses.

Quick Start

Use the kaggle-benchmarks skill to create a new benchmark task that evaluates a model's math skills.

Frequently Asked Questions about kaggle-benchmarks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I create and evaluate AI benchmarks for large language models?▼

You can create and evaluate AI benchmarks by defining custom tasks using decorators and structured outputs, then asserting the correctness of the model's responses. This ensures reproducible and robust model evaluation.

How do I define custom benchmark tasks to test an AI model's math skills?▼

Define custom benchmark tasks by using decorators and structured outputs to specify the required interactions and assertions. This allows you to accurately test specific capabilities like mathematical question answering.

Can I use this to interact with various large language models and assert output correctness?▼

Yes, you can interact with various large language models and assert the correctness of their outputs. The skill supports model evaluation through structured task definitions and validation assertions.

Do I need the kaggle_benchmarks library to execute and evaluate benchmark tasks?▼

Yes, you need the kaggle_benchmarks library installed to execute and evaluate benchmark tasks. The library handles the task execution and model assessment workflows required for robust evaluation.

What is the best way to ensure reproducibility when assessing AI model performance?▼

The best way to ensure reproducibility in AI model assessment is to use structured benchmark task definitions with specific assertions. This approach standardizes the evaluation criteria across different large language models.

Can I evaluate a model's tool usage and dataset performance within a benchmark task?▼

Yes, you can evaluate a model's tool usage and dataset performance within a benchmark task. The skill supports tool usage and dataset evaluation to comprehensively test and assert correctness of outputs.