ds-utils-preprocess

Visualize feature distributions, correlations, and interactions from data science preprocessing workflows.

37|7|Updated Jun 24, 2018

Search Tags:#statistics #preprocessing #pca #visualize #correlations #dendrogram #feature-distribution

Ready to run

npx skills add https://github.com/idanmoradarthas/DataScienceUtils --skill ds-utils-preprocess

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: ds-utils-preprocess
Source: https://github.com/idanmoradarthas/DataScienceUtils/tree/main/skills/preprocess
Command: npx skills add https://github.com/idanmoradarthas/DataScienceUtils --skill ds-utils-preprocess

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Visualize preprocessing visualizations and statistics for feature analysis in data science workflows.

Core Features & Use Cases

visualize_feature to inspect a single feature's distribution based on its data type
visualize_correlations to visualize a correlation matrix as a heatmap
plot_correlation_dendrogram to reveal hierarchical relationships among features
plot_features_interaction to explore joint feature relationships
plot_pca_explained_variance to assess PCA explained variance
get_correlated_features to identify highly correlated feature pairs
extract_statistics_dataframe_per_label to summarize statistics by category
compute_mutual_information to measure the dependency between features and a target label

Quick Start

Invoke ds_utils.preprocess.visualization and ds_utils.preprocess.statistics on your DataFrame to generate plots and summary statistics.

Frequently Asked Questions about ds-utils-preprocess

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I visualize feature distributions for different data types in a DataFrame?▼

You can visualize feature distributions by invoking preprocessing functions on your DataFrame to inspect a single feature's distribution based on its data type. This helps understand data shape and identify outliers during model preparation.

What is the best way to visualize a correlation matrix and identify highly correlated features?▼

You can visualize correlations by generating a correlation heatmap and then identifying highly correlated feature pairs. This reveals linear relationships between variables and helps reduce multicollinearity before modeling.

How does a correlation dendrogram help with feature analysis?▼

A correlation dendrogram reveals hierarchical relationships among features by clustering them based on correlation distance. This visualization helps identify groups of redundant variables for feature selection and dimensionality reduction.

How do I measure the dependency between features and a target label?▼

You can measure feature dependency by computing mutual information between features and a target label. This quantifies how much information the feature provides about the target, supporting effective feature selection beyond linear correlations.

Can I assess PCA explained variance and plot feature interactions without external dependencies?▼

Yes, you can assess PCA explained variance and plot feature interactions directly. The skill operates with no external dependencies, allowing you to evaluate principal components and explore joint feature relationships seamlessly.

How do I summarize statistics by category for a labeled dataset?▼

You can summarize statistics by category by extracting a statistics dataframe per label. This computes summary metrics for features grouped by their target category, providing detailed insights into class-specific data distributions.