ds-utils-transformers

Encode multi-label sets and text into sklearn pipeline features.

37|7|Updated Jun 24, 2018
One-click install
npx skills add https://github.com/idanmoradarthas/DataScienceUtils --skill ds-utils-transformers
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ds-utils-transformers
Source: https://github.com/idanmoradarthas/DataScienceUtils/tree/main/skills/transformers
Command: npx skills add https://github.com/idanmoradarthas/DataScienceUtils --skill ds-utils-transformers

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Provides Sklearn-compatible transformers to integrate multi-label binarization and sentence embeddings into pipelines, enabling feature extraction and preprocessing in a consistent interface.

Core Features & Use Cases

  • MultiLabelBinarizerTransformer: wraps sklearn's MultiLabelBinarizer so pipelines get get_feature_names_out and dense matrices.
  • SentenceEmbeddingTransformer: wraps sentence-transformers models for use in pipelines with lazy loading and NaN handling.
  • Use Case: Build a pipeline to binarize tags and then feed embeddings into a classifier, or combine text features with numeric ones using ColumnTransformer.

Quick Start

Create a Pipeline using MultiLabelBinarizerTransformer for your label column and, when processing text data, append SentenceEmbeddingTransformer to produce embeddings for model training.

Frequently Asked Questions about ds-utils-transformers

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I integrate multi-label binarization into an scikit-learn pipeline?

Multi-label binarization integrates into scikit-learn pipelines using a wrapper transformer that exposes get_feature_names_out and outputs dense matrices. It wraps sklearn's MultiLabelBinarizer to ensure compatibility within standard pipeline and ColumnTransformer workflows.

Can I use sentence embeddings in a scikit-learn ColumnTransformer?

Sentence embeddings can be used in a scikit-learn ColumnTransformer via a wrapper around sentence-transformers models. This wrapper supports lazy loading and NaN handling, allowing text features to be combined with numeric ones inside standard pipelines.

What's the best way to combine text embeddings and multi-label features for classification?

Combining text embeddings and multi-label features for classification is done by placing a binarizer transformer and an embedding transformer in a pipeline or ColumnTransformer. This setup feeds both binary label matrices and sentence embeddings directly into a classifier.

Does scikit-learn's MultiLabelBinarizer support get_feature_names_out for pipelines?

Scikit-learn's MultiLabelBinarizer does not natively support get_feature_names_out for pipelines. A custom wrapper is required to expose this method and ensure the transformer outputs dense matrices compatible with pipeline feature extraction.

How do sentence embeddings handle missing values or lazy loading in sklearn pipelines?

Sentence embeddings handle missing values and lazy loading in sklearn pipelines through a dedicated sentence-transformers wrapper. It manages NaN values during transformation and defers model loading until needed, maintaining standard scikit-learn API compatibility.

When do I need custom transformers for multi-label sets and text in NLP pipelines?

Custom transformers for multi-label sets and text are needed in NLP pipelines when extracting machine-learning-ready features. They provide a consistent scikit-learn interface for generating binary label matrices and sentence embeddings required by downstream classifiers.