iterate-ml-experiment

Orchestrates the propose-approve-implement-record loop for ML experiments via journal design notes.

1|Updated Jun 22, 2026
One-click install
npx skills add https://github.com/probabl-ai/ibm-workshop --skill iterate-ml-experiment-probabl-ai
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: iterate-ml-experiment
Source: https://github.com/probabl-ai/ibm-workshop/tree/main/.bob/skills/iterate-ml-experiment
Command: npx skills add https://github.com/probabl-ai/ibm-workshop --skill iterate-ml-experiment-probabl-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? ML experimentation often loses track of why experiments were run, what was tried, and what the results mean. This Skill enforces a disciplined iteration loop where every experiment script is preceded by an approved design note and every finished run is recorded in a durable journal, keeping the project history reproducible and auditable. ## Core Features & Use Cases - Design-note gating: Blocks creation of experiments/NN_*.py until a matching journal/NN_*.md design note is drafted and explicitly approved by the user. - Mode-driven workflow: Routes each session to bootstrap, propose, record-outcome, compare, goal-pivot, abandoned, or re-run modes based on journal state and user signals. - Sourcing strategies: Sources the next experiment from skore report audits, user-provided articles or issues, synthesized candidate ideas, or a numbered Backlog. - Use Case: In a cross-validated classification project, the user asks "what's next"; the Skill reads journal/JOURNAL.md, presents the sourcing menu, drafts a design note, and only after approval dispatches the build, evaluate, and test pipeline skills. ## Quick Start Ask the assistant to propose the next ML experiment based on the current journal and backlog in this workspace.

Frequently Asked Questions about iterate-ml-experiment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I propose the next ML experiment in a journaled workspace?

Read journal/JOURNAL.md, then present the sourcing menu with options skore, user, my-pick, or a Backlog row number. After the user picks, draft a design note at journal/NN_short_name.md and wait for explicit approval before writing any experiment script.

How to record results after an ML experiment run finishes?

Dispatch the audit pipeline to generate a digest with headline metrics, then fill the design note's Status block with state, headline result, and implication. Update the JOURNAL.md History row, prune resolved Backlog items, and verify all smoke tests pass before marking done.

Can I create an experiment script before the design note is approved?

No. The skill forbids creating or editing experiments/NN_*.py until the matching journal/NN_*.md design note exists and the user has explicitly approved it. Ambiguous responses like 'I guess' trigger a re-ask rather than silent approval.

What happens on the first session in a new ML workspace?

The skill enters bootstrap mode: it rewrites JOURNAL.md from the template, derives a goal from the data README, optionally runs EDA, and auto-drafts a baseline design note. Config gates for package name, environment manager, tabular library, and skore mode still fire.

Does the skill support comparing past experiments?

Yes, as a read-only mode. It surfaces headline results side by side from JOURNAL.md History and routes deeper per-report reads to the evaluation skill one stem at a time. Programmatic multi-stem statistical comparison is out of scope for v1.

When should an experiment be marked abandoned instead of done?

Mark abandoned only on explicit user decision via a structured question offering abandon, defer, or run now. The Status block requires a one-line reason, the headline becomes n/a with that reason, and the History row stays for provenance.