skills.rest
ruvnet
ruvnetCOMMUNITY
Explore all skills by this author →
Website𝕏@ruvGitHub

harness-bench

Creates and verifies JSON bench suites for scoring harness evolution variants.

70.1k|8.4k|Updated Jun 2, 2025
Search Tags:#test scaffolding#CI validation#harness evolution#fitness scoring#metaharness#bench suite#evaluation corpus
One-click install
npx skills add https://github.com/ruvnet/claude-flow --skill harness-bench
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: harness-bench
Source: https://github.com/ruvnet/claude-flow/tree/main/plugins/ruflo-metaharness/skills/harness-bench
Command: npx skills add https://github.com/ruvnet/claude-flow --skill harness-bench

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires @metaharness/darwin, and includes scripts (resource) components.

What problem does it solve?

Evolving a harness against a repo's natural test suite produces incomparable fitness scores when tests are flaky, slow, or change between commits. This Skill scaffolds a fixed JSON bench suite from a repo's test corpus and verifies suite well-formedness, so evolution runs are scored against a stable, deterministic corpus.

Core Features & Use Cases

  • Bench suite creation: bench create <repo> derives a suite of {input, expectedOutput, weight} tasks from the repo's existing test cases, written to .metaharness/bench/suite.json by default.
  • Suite verification: bench verify <suite.json> checks that every task is well-formed, exiting 1 on malformed suites — ideal for CI checks on PRs that touch the suite.
  • Graceful degradation: when @metaharness/darwin is absent, emits a {degraded: true} payload and exits 0 instead of failing.
  • Use Case: Fork a harness to a new domain, copy and edit the bench suite to retarget evaluation, then run harness-evolve --bench suite.json so champion fitness stays comparable to the parent fork.

Quick Start

Create a bench suite from my repository's test corpus and verify that the generated suite.json is well-formed.

Frequently Asked Questions about harness-bench

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a bench suite from a repo's test corpus?▼

Run the create operation with the repo path: it shells to `metaharness-darwin bench create <repo>`, which derives tasks of the form `{input, expectedOutput, weight}` from existing test cases. The default output is `<repo>/.metaharness/bench/suite.json`.

How do I validate a bench suite JSON file in CI?▼

Run the verify operation with the suite path, which calls `metaharness-darwin bench verify <suite.json>`. It exits 1 if any task is malformed, making it a cheap (~5s) gate on pull requests that modify the suite.

Why use a bench suite instead of npm test for harness evolution?▼

A bench suite is a fixed evaluation corpus decoupled from the repo's natural tests, so champion fitness stays comparable across commits or across forks. This matters when npm test is flaky, slow, or undersized for scoring evolution variants.

What happens if @metaharness/darwin is not installed?▼

The skill degrades gracefully: it emits a `{degraded: true, reason: 'metaharness-darwin-not-available'}` payload and exits 0 rather than failing, so pipelines can detect the missing dependency without breaking.

What do the harness-bench exit codes mean?▼

Exit 0 means success or degraded operation when Darwin is absent. Exit 1 means the verify operation found a malformed suite. Exit 2 indicates a configuration error or a failed upstream invocation.

Related Skills

Explore More Software Engineering Skills→
hw-native-syshw-native-sys
30

llvm-test-tool-fallback

Locate missing lit or FileCheck tools in LLVM environments.

Official
Basic
514-labs514-labs
1

dec-bench-run

Build and run DEC Bench scenarios, inspect results, and open the audit UI.

Official
Intermediate
WangK0221WangK0221

Evolution Dashboard Skill

Generate a visual dashboard tracking AI evolution progress and learning milestones.

Community
Intermediate
selcukyucelselcukyucel
1

prompt-test

Run a prompt against test inputs and score outputs with a binary rubric.

Community
Advanced