baseline

Read the source paper and repo first, identify the real run and evaluation entrypoints, then set up the environment with uv, run one bounded smoke test, and launch the real run. Verify the metrics against the paper before accepting the baseline.

Prefer attach or import when a verified reusable baseline already exists and its provenance, dataset, and metric contract match the quest. Reproduce only when no credible baseline exists, and repair only for bounded failures in an existing line.

A baseline is verified when the run finished, metrics came from the intended dataset and split, metric definitions match the quest contract, and deviations are recorded. The comparability contract must state task, dataset version, split, evaluation path, metric keys, and source commit.

Divergence usually comes from implementation mismatch, environment differences, data or split mismatch, or expected stochastic variance. Verification must separate these causes explicitly and classify the outcome as verified_match, verified_close, verified_diverged, or broken.

Python baseline setup is standardized around uv. Use uv sync when a uv.lock or solid pyproject.toml exists, otherwise uv venv and uv pip install, and run commands through uv run for reproducibility.

Record the blocked class explicitly, such as missing_source, environment_infeasible, run_failed, or verification_failed, along with what was tried and log paths. Then route to attach, import, retry, repair, or an explicit waiver via artifact.waive_baseline.