harbor-adapter-creator

Create Harbor benchmark adapters from external datasets in standardized task format.

Updated Mar 27, 2026

Search Tags:#Harbor #benchmark #conversion #adapter #dataset

Ready to run

npx skills add https://github.com/qiushiyan/harbor-eval-experiments --skill harbor-adapter-creator

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: harbor-adapter-creator
Source: https://github.com/qiushiyan/harbor-eval-experiments/tree/main/.claude/skills/harbor-adapter-creator
Command: npx skills add https://github.com/qiushiyan/harbor-eval-experiments --skill harbor-adapter-creator

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires adapter-class, data-structures, json, python, harbor-cli, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill addresses the challenge of integrating external benchmark datasets into Harbor's standardized task format, allowing for parity experiments, dataset registration, and debugging adapter validation issues.

Core Features & Use Cases

Adapter Creation: Generate Harbor benchmark adapters for converting external datasets.
Parity Experiments: Facilitate parity testing with original benchmarks.
Dataset Registration: Register datasets with Harbor registry for better accessibility and management.
Use Case: When migrating a benchmark like SimpleQA or GAIA to Harbor, use this Skill to ensure tasks are in the correct format for evaluation and experimentation.

Quick Start

Run the following command to create a new adapter for a benchmark:

harbor adapters init my-benchmark

Frequently Asked Questions about harbor-adapter-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I convert external benchmark datasets to the Harbor task format?▼

To convert external benchmark datasets to the Harbor task format, generate a Harbor benchmark adapter. This standardizes external data into the correct task directory format for evaluation and experimentation.

What is the process for registering a custom dataset with the Harbor registry?▼

Registering a custom dataset with the Harbor registry requires creating a benchmark adapter. This converts external data into the standardized task format, enabling better accessibility and management within Harbor.

How do I run parity experiments when migrating benchmarks to Harbor?▼

Running parity experiments when migrating benchmarks to Harbor involves creating an adapter for the external dataset. This ensures tasks are in the correct format for accurate parity testing against original benchmarks.

Do I need Python to create Harbor adapters for external datasets?▼

Yes, you need Python to create Harbor adapters. Python is required for executing the adapter scripts and validating the converted external datasets against the standardized task directory format.

How do I initialize a new adapter for a benchmark using the harbor CLI?▼

To initialize a new adapter for a benchmark using the harbor CLI, run the command `harbor adapters init my-benchmark`. This generates the foundational adapter structure for the external dataset.