ds-utils-preprocess

Visualize feature distributions, correlations, and interactions from data science preprocessing workflows.

37|7|Updated Jun 24, 2018
One-click install
npx skills add https://github.com/idanmoradarthas/DataScienceUtils --skill ds-utils-preprocess
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ds-utils-preprocess
Source: https://github.com/idanmoradarthas/DataScienceUtils/tree/main/skills/preprocess
Command: npx skills add https://github.com/idanmoradarthas/DataScienceUtils --skill ds-utils-preprocess

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Visualize preprocessing visualizations and statistics for feature analysis in data science workflows.

Core Features & Use Cases

  • visualize_feature to inspect a single feature's distribution based on its data type
  • visualize_correlations to visualize a correlation matrix as a heatmap
  • plot_correlation_dendrogram to reveal hierarchical relationships among features
  • plot_features_interaction to explore joint feature relationships
  • plot_pca_explained_variance to assess PCA explained variance
  • get_correlated_features to identify highly correlated feature pairs
  • extract_statistics_dataframe_per_label to summarize statistics by category
  • compute_mutual_information to measure the dependency between features and a target label

Quick Start

Invoke ds_utils.preprocess.visualization and ds_utils.preprocess.statistics on your DataFrame to generate plots and summary statistics.

Frequently Asked Questions about ds-utils-preprocess

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I visualize feature distributions for different data types in a DataFrame?

You can visualize feature distributions by invoking preprocessing functions on your DataFrame to inspect a single feature's distribution based on its data type. This helps understand data shape and identify outliers during model preparation.

What is the best way to visualize a correlation matrix and identify highly correlated features?

You can visualize correlations by generating a correlation heatmap and then identifying highly correlated feature pairs. This reveals linear relationships between variables and helps reduce multicollinearity before modeling.

How does a correlation dendrogram help with feature analysis?

A correlation dendrogram reveals hierarchical relationships among features by clustering them based on correlation distance. This visualization helps identify groups of redundant variables for feature selection and dimensionality reduction.

How do I measure the dependency between features and a target label?

You can measure feature dependency by computing mutual information between features and a target label. This quantifies how much information the feature provides about the target, supporting effective feature selection beyond linear correlations.

Can I assess PCA explained variance and plot feature interactions without external dependencies?

Yes, you can assess PCA explained variance and plot feature interactions directly. The skill operates with no external dependencies, allowing you to evaluate principal components and explore joint feature relationships seamlessly.

How do I summarize statistics by category for a labeled dataset?

You can summarize statistics by category by extracting a statistics dataframe per label. This computes summary metrics for features grouped by their target category, providing detailed insights into class-specific data distributions.