experiment-harness

Name: experiment-harness
Availability: InStock
Author: cyberAlchemyAI

Initialize and run repeatable experiment harnesses for Arcanum spells and sigils.

20|3|Updated May 14, 2026

Search Tags:#validation #telemetry #experiments #harness #arcanum #spell-sigil

Ready to run

npx skills add https://github.com/cyberAlchemyAI/Arcanum --skill experiment-harness

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: experiment-harness
Source: https://github.com/cyberAlchemyAI/Arcanum/tree/main/.claude/skills/experiment-harness
Command: npx skills add https://github.com/cyberAlchemyAI/Arcanum --skill experiment-harness

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires jq, codex, and includes scripts (resource) components.

What problem does it solve?

The Experiment Harness provides a repeatable, profile-aware workflow to validate Arcanum spells and sigils by coordinating prompts, native skill runs, observed outputs, and timestamped reports.

Core Features & Use Cases

Initializes a development harness structure with fixtures, regimes, prompts, and outputs.
Executes bounded native skill/subagent runs or explicit legacy runtime adapters to gather real evidence.
Produces observable artifacts (validation, reports, and telemetry) and supports loop-based improvement and observability.

Quick Start

Initialize the harness for your artifact and start the first validation loop.

Frequently Asked Questions about experiment-harness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I set up a repeatable experiment harness for validating prompts?▼

To validate spell outputs, the experiment harness executes bounded native skill runs and legacy runtime adapters, capturing real runtime outputs and validation evidence. It then generates timestamped reports with optional observability telemetry for analysis.

What is a profile-aware workflow for collecting runtime validation evidence?▼

A profile-aware workflow coordinates prompts, native skill runs, and observed outputs across specific regimes. It produces observable artifacts like validation reports and telemetry to verify runtime behavior.

Do I need jq and codex to run experiment validation loops?▼

Yes, jq and codex are required dependencies to operate the experiment harness. They provide the host runtime adapters and processing logic required to execute bounded runs and parse validation artifacts.

Can I use a contract at SKILL.md to initialize a development harness?▼

Yes, a contract at SKILL.md or README.md is required to initialize the development harness. The harness uses this contract to identify the artifact and structure the initial prompts, regimes, and output directories.

What's the best way to loop for stability when running native skill experiments?▼

Use the harness's built-in loop-based improvement workflow to iterate bounded native skill runs against your fixtures. This process validates outputs repeatedly to ensure consistent runtime behavior and generates timestamped reports.

When should I not use an experiment harness for runtime validation?▼

Avoid using an experiment harness without a defined contract at SKILL.md or access to host runtime adapters. Without structured prompts, regimes, and fixtures, the harness cannot execute bounded runs or generate validation evidence.