ds-baseline

Establishes verified research baselines through attach, import, reproduce, or repair routes.

1.1k|116|Updated Feb 26, 2026
One-click install
npx skills add https://github.com/OpenLAIR/dr-claw --skill ds-baseline
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ds-baseline
Source: https://github.com/OpenLAIR/dr-claw/tree/main/skills/ds-baseline
Command: npx skills add https://github.com/OpenLAIR/dr-claw --skill ds-baseline

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Research experiments need a trustworthy reference point before any comparison is meaningful, but reproducing or validating baselines is error-prone, often producing fabricated metrics, unverified runs, or incomparable results. This Skill enforces a disciplined workflow for attaching, importing, reproducing, or repairing a baseline so downstream experiment and writing stages can compare against it fairly.

Core Features & Use Cases

  • Four baseline routes: Attach an existing reusable baseline, import a baseline package, reproduce from a source paper and repo, or repair a broken baseline, with a fast-path default for cheap validation.
  • Mandatory verification and comparability contract: Classifies outcomes as verified_match, verified_close, verified_diverged, or broken, and records task, dataset, split, metric keys, and deviations in a canonical metric_contract.json.
  • Durable artifacts and gating: Produces PLAN.md, CHECKLIST.md, verification notes, and a confirmed baseline gate via artifact.confirm_baseline, or an explicit waiver, before downstream work proceeds.
  • Use Case: A research quest needs to compare a new method against a published model. The Skill reads the source paper and repo, sets up the environment with uv, runs a bounded smoke test then the real baseline run via bash_exec, verifies the metrics against the paper, and confirms the baseline for the experiment stage.

Quick Start

Ask the agent to establish and verify a baseline for the current quest by reproducing the specified paper's repository and confirming the comparison metrics.

Frequently Asked Questions about ds-baseline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reproduce a baseline from a research paper?

Read the source paper and repository first, then create PLAN.md and CHECKLIST.md defining the route, command path, and acceptance condition. Set up the Python environment with uv, run one bounded smoke test, then launch the real run and verify the metrics against the paper before acceptance.

When should I attach or import a baseline instead of reproducing it?

Prefer attach or import when a verified reusable baseline already exists or a baseline package is available, since reuse is cheaper than reproduction. Reproduce only when no credible baseline exists, and repair only when an existing baseline has a bounded, identifiable failure.

What makes a baseline verified and comparable for downstream experiments?

A baseline is comparable when the task identity, dataset version, split contract, evaluation path, metric keys, and source commit are explicit and recorded in metric_contract.json. Verification classifies the result as verified_match, verified_close, verified_diverged, or broken before acceptance.

Why does baseline reproduction fail and how should it be handled?

Failures are classified as missing_source, missing_code, environment_infeasible, command_unknown, run_failed, or verification_failed. One autonomous fix per failure class is the normal bound; recurring failures should be recorded as blocked with a next-step decision rather than retried endlessly.

Can a quest proceed without a baseline?

Yes, but only through an explicit waiver recorded via artifact.waive_baseline with a clear reason. Without a confirmed or waived baseline gate, comparison-heavy downstream stages like experiment should not proceed.