moai-workflow-research

Define an eval suite in YAML with test inputs and binary pass/fail criteria, record a baseline measurement, then run experiments in an isolated worktree. Compare results against the baseline and pass canary regression checks before proposing a merge.

A binary eval is a yes/no question applied to each test output, such as whether the response meets a defined criterion. Each eval specifies what passing and failing look like, with weights marking criteria as must_pass or nice_to_have.

Five layers protect the system: FrozenGuard blocks constitution edits, experiments run in isolated worktrees, canary checks catch regressions, rate limits cap experiments per session, and human approval is required before merging.

No. Constitution files are frozen and cannot be modified by the research workflow. Amendments must be proposed through the human approval channel instead of direct edits.

The rate limiter caps the number of experiments per session and per week to prevent runaway self-modification. Batch your experiments or wait for the next window if you hit the limit.