audit-ml-pipeline

Executes read-only jupytext audit files against skore reports and renders markdown digests.

1|Updated Jun 22, 2026
One-click install
npx skills add https://github.com/probabl-ai/ibm-workshop --skill audit-ml-pipeline-probabl-ai
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: audit-ml-pipeline
Source: https://github.com/probabl-ai/ibm-workshop/tree/main/.bob/skills/audit-ml-pipeline
Command: npx skills add https://github.com/probabl-ai/ibm-workshop --skill audit-ml-pipeline-probabl-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires ipython, matplotlib, pandas, and includes scripts (resource) and references (resource) components.

What problem does it solve? After an ML experiment writes a skore report, extracting its checks and metrics for narrative review usually means ad-hoc scratch scripts that risk mutating the project. This Skill standardizes that step: one durable, read-only audit file per experiment, executed by a bundled in-process runner that streams a markdown digest of each cell's output. ## Core Features & Use Cases - Read-only report auditing: Opens the skore Project, loads a report by id, and renders report.checks.summarize().frame() and report.metrics.summarize().frame() — never calling skore.evaluate(...) or project.put(...). - In-process cell runner: scripts/run_cells.py executes jupytext # %% files via IPython's InteractiveShell.run_cell, capturing stdout, stderr, and last-expression reprs into a markdown digest streamed to stdout or written to scratch/audit/<stem>/audit.md. - Four-way stem pairing: Each audit file aligns 1:1 with journal/NN_*.md, experiments/NN_*.py, and tests/smoke/test_NN_*.py, keeping the audit trail durable in git. - Use Case: After experiment 02_target_transform finishes, ask the agent to audit it — it places audit/02_target_transform.py from the template, runs the bundled runner, and surfaces the checks and metrics digest for the JOURNAL update. ## Quick Start Ask the agent to audit experiment 02 and show its checks and metrics digest from the stored skore report.

Frequently Asked Questions about audit-ml-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit a finished ML experiment's skore report?

Place an audit file at audit/NN_<short_name>.py from the bundled template, copying the Project init block from the matching experiment script. Then run the bundled run_cells.py runner, which streams a markdown digest of the checks and metrics summaries to stdout.

How do I load a skore report by id instead of key?

project.get() looks up reports by id, not key. In hub mode, derive the id from the URL printed by project.put() (plural path segment becomes singular, e.g. cross-validations/42 becomes skore:report:cross-validation:42); in local mode, read the id column from project.summarize().

Why does my audit cell show <Display object at 0x...> instead of values?

The skore summarize() accessors return Display objects whose plain repr carries no values. Chain .frame() — report.checks.summarize().frame() — to get a pandas DataFrame whose repr the runner can render into the digest.

Can the audit file re-run evaluate or put to refresh a report?

No. Audit files are strictly read-only: skore.evaluate and project.put are forbidden because they duplicate reports under the same key and pollute summarize(). A KeyError from get() means the lookup shape is wrong, not that the report is missing.

What happens if IPython is not installed when running the audit?

The runner fails with ModuleNotFoundError because the agent feature is missing. The skill stops and delegates installation to the python-env-manager skill rather than installing packages itself or faking output with print calls.