Convert external benchmarks into the BenchFlow framework with routing and parity verification.
npx skills add https://github.com/benchflow-ai/benchflow --skill adapter-benchflow-ai
Please help me install this Agent Skill. Skill: adapter Source: https://github.com/benchflow-ai/benchflow/tree/main/.agents/skills/adapter Command: npx skills add https://github.com/benchflow-ai/benchflow --skill adapter-benchflow-ai
This Skill solves the fragmentation of benchmark formats by providing a structured, verifiable path to porting external benchmarks into the BenchFlow ecosystem.
Use the adapter skill to classify the benchmark source at the provided path and generate the necessary scaffold for conversion.
High-intent search queries and answers about installing and using this skill.
To convert external benchmarks into BenchFlow, you use a multi-layer routing process that classifies the source and generates a deterministic adapter. This scaffolds the directory and ensures high-fidelity reproduction of the original benchmark behavior.
Benchmark parity verification automates side-by-side and reward-distribution checks to ensure converted benchmarks behave exactly like their originals. It validates that the structural conversion maintains the original scoring logic and task format.
You standardize diverse evaluation task formats by routing them through native, translated, or as-is layers. This classification ensures the most efficient integration path and generates structural validation for the specific benchmark.
Yes, automated adapters can process as-is benchmark formats through the routing layer. The system classifies the benchmark source and applies the appropriate integration layer to facilitate verifiable porting into the BenchFlow environment.
The best way to scaffold a benchmark converter directory is to use an automated generation process that classifies the source format and creates the necessary structural files. This ensures the conversion path is deterministic and verifiable.
If benchmark scoring logic changes after conversion, the structural validation or parity gate may have failed. Running side-by-side parity checks verifies that the deterministic adapter accurately reproduces the original behavior.
tentacle-proInitialize Obsidian vault scaffolds with PARA folders, templates, and AGENTS.md guardrails.
siddvohEnforce parity across model tier, temperature, max_tokens, system prompts, seeds, and cost before executing AI-model comparisons.
pmcdowallEvaluate large language models against standardized benchmarks with lm-evaluation-harness.
qiushiyanCreate Harbor benchmark adapters from external datasets in standardized task format.
