skills.rest
probabl-ai
probabl-aiOFFICIAL
Explore all skills by this author →
Website𝕏@probabl_aiGitHub

build-ml-pipeline

Declare machine learning pipelines as skrub DataOps graphs from data source to predictor.

1|Updated Jun 22, 2026
Search Tags:#machine learning#feature engineering#pipeline#dataops#data leakage#scikit-learn#skrub
One-click install
npx skills add https://github.com/probabl-ai/ibm-workshop --skill build-ml-pipeline-probabl-ai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: build-ml-pipeline
Source: https://github.com/probabl-ai/ibm-workshop/tree/main/.bob/skills/build-ml-pipeline
Command: npx skills add https://github.com/probabl-ai/ibm-workshop --skill build-ml-pipeline-probabl-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires skrub, scikit-learn, skore, and includes references (resource) components.

What problem does it solve? It prevents structural mistakes when declaring ML pipelines — leakage from stateless misuse, late X-marker placement breaking cold-start prediction, and bare sklearn.Pipeline usage — by enforcing a skrub DataOps graph declaration with explicit layering rules. ## Core Features & Use Cases - Skrub DataOps declaration: Roots pipelines on skrub.var(...) source identifiers, attaches stateless steps via .skb.apply_func and stateful estimators via .skb.apply, and stops at the unfit learner object. - Three-layer topology rules: Separates sources (Layer 1), the predict grid with mark_as_X/mark_as_y (Layer 2), and post-marker feature engineering (Layer 3), with worked examples for IID and history-dependent cases. - Leakage and reproducibility guards: Enforces the stateless-vs-stateful decision rule, forbids splitter/tuning/persistence code in pipeline declarations, and defines three options for extending shared modules without breaking prior experiments. - Use Case: When adding a new feature step or estimator to pipeline.py in a tabular ML project, consult this Skill to place the X marker correctly, wire split_kwargs for grouped CV, and keep prior experiments reproducible. ## Quick Start Ask the AI to declare a skrub DataOps pipeline from a data directory to a HistGradientBoosting predictor, following the pre-flight checklist before writing any code.

Frequently Asked Questions about build-ml-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build an ML pipeline with skrub DataOps?▼

Root the graph on skrub.var with a source identifier, load data via .skb.apply_func, mark X and y on the loaded frame, then chain feature steps and attach the final estimator with .skb.apply. Finish with .skb.make_learner() to get the unfit learner.

skrub DataOps vs sklearn Pipeline: which should I use?▼

Use a skrub DataOps graph as the top-level declaration instead of a bare sklearn.Pipeline. It supports source binding, predict-time replay via environment dicts, and hyperparameter knobs via skrub.choose_from, while sklearn.Pipeline remains usable only as an inner component.

When should I use apply_func versus apply in skrub?▼

Use .skb.apply_func for stateless operations where a row's output depends only on that row, and .skb.apply for stateful steps that learn statistics from training data. If calling the function on the full frame would leak test information, it must be an estimator.

Why does my skrub pipeline fail at predict time with cold-start rows?▼

This happens when mark_as_X is placed after cross-row feature steps like lags or rolling windows. Move the X marker upstream of every cross-row step and pass the history DataOp as an extra apply_func argument so predict-grid rows see real history values.

Can I use skrub.X and skrub.y as pipeline roots?▼

No, skrub.X and skrub.y are discouraged as graph roots because they bake the marker at the source and force pre-loaded data binding. Root on skrub.var with a source identifier and mark X and y inside the graph at the split point.

Does this skill handle hyperparameter tuning or cross-validation?▼

No, it stops at the declared unfit learner. Splitter selection and evaluation belong to a separate evaluation skill, though you can expose tuning knobs with skrub.choose_from and wire split_kwargs groups at the X marker.

Related Skills

Explore More Data & Analytics Skills→
vathymutvathymut

data-science-python-stack

Set up a Python data science stack with scikit-learn, PyTorch, and Polars.

Community
Advanced
vathymutvathymut

evaluate-ml-pipeline

Evaluate sklearn-compatible machine learning models with cross-validation and performance reports.

Community
Advanced
Yash-AwasthiYash-Awasthi

python-api

Look up installed Python package APIs via inspect, pydoc, LSP, and versioned docs, then cache findings.

Community
Advanced
idanmoradarthasidanmoradarthas
37

ds-utils-transformers

Encode multi-label sets and text into sklearn pipeline features.

Community
Intermediate