skills.rest
TheodoreGalanos
TheodoreGalanosCOMMUNITY
Explore all skills by this author →
Website𝕏@TheodoreGalanosGitHub

create-dataset

Create versioned benchmark datasets through guided template discovery and configuration.

35|5|Updated Apr 11, 2026
Search Tags:#automation#templates#validation#benchmark#dataset#instances
One-click install
npx skills add https://github.com/TheodoreGalanos/aec-bench --skill create-dataset
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: create-dataset
Source: https://github.com/TheodoreGalanos/aec-bench/tree/main/src/aec_bench/init/skill_data/create-dataset
Command: npx skills add https://github.com/TheodoreGalanos/aec-bench --skill create-dataset

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Create versioned, immutable benchmark datasets through guided discovery and configuration. Datasets are the formal "this IS the benchmark" artifact that sits between template generation and experiment execution.

Core Features & Use Cases

  • Guided template discovery and task counting to scope templates available for dataset creation.
  • Generate parameterised task instances from selected templates and freeze results into an immutable manifest.
  • Validate integrity, publish or export the finalized dataset, and reference it in experiments.

Quick Start

Follow the guided prompts to create a versioned dataset by discovering templates, generating instances, and publishing the manifest.

Frequently Asked Questions about create-dataset

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create versioned benchmark datasets for AI evaluation?▼

Versioned benchmark datasets are created through guided template discovery, parameterized task instance generation, and dataset freezing into an immutable manifest to establish a formal evaluation artifact.

How do I generate task instances from templates for a benchmark dataset?▼

Task instances are generated by discovering available templates, configuring parameters, and freezing the resulting instances into an immutable manifest to lock the benchmark dataset configuration.

What is dataset freezing and why is it needed for benchmark evaluation?▼

Dataset freezing locks generated task instances into an immutable manifest, ensuring the benchmark dataset remains unchanged and providing a stable, verifiable reference for experiment execution.

Can I validate dataset integrity before publishing a benchmark evaluation set?▼

Dataset integrity is verified after freezing the manifest and before publishing or exporting the finalized benchmark dataset, ensuring the evaluation set remains consistent and reliable for experiments.

What is the best way to scope templates available for dataset creation?▼

Template scoping is handled through guided discovery and task counting, allowing you to evaluate available templates before generating parameterized task instances for your benchmark dataset.

Do I need existing templates to start building a versioned benchmark dataset?▼

Existing templates are required to start the process, as the workflow relies on discovering available templates to generate and freeze parameterized task instances into the dataset manifest.

Related Skills

Explore More Data & Analytics Skills→
LeonChaoXLeonChaoX
783

pytdc

Load PyTDC datasets with benchmark splits for drug discovery modeling.

Community
Advanced
surus-latsurus-lat
8

push-to-latamboard

Merge local benchmark results into the HuggingFace LatamBoard leaderboard dataset.

Official
Advanced
floflo777floflo777
33

rag-eval

Evaluate RAG systems using retrieval, generation, and latency metrics.

Community
Intermediate
jfrogjfrog
5

jfrog-curation-onboarding

Automate JFrog Curation setup and governance on existing Platform instances.

Official
Advanced