harbor-adapter-creator

What problem does it solve?

This Skill addresses the challenge of integrating external benchmark datasets into Harbor's standardized task format, allowing for parity experiments, dataset registration, and debugging adapter validation issues.

Core Features & Use Cases

• Adapter Creation: Generate Harbor benchmark adapters for converting external datasets.
• Parity Experiments: Facilitate parity testing with original benchmarks.
• Dataset Registration: Register datasets with Harbor registry for better accessibility and management.
• Use Case: When migrating a benchmark like SimpleQA or GAIA to Harbor, use this Skill to ensure tasks are in the correct format for evaluation and experimentation.

Quick Start

Run the following command to create a new adapter for a benchmark:

harbor adapters init my-benchmark