feature-analysis

Feature selection for a binary finance dataset requires analyzing predictive power, stability, and redundancy. This process computes IV, PSI stability, and Pearson correlation across all features to identify weak variables and output four distinct modeling feature schemes.

Calculating IV and PSI involves splitting your dataset into baseline and comparison segments to measure predictive value and population stability. The analysis ranks features by IV, identifies significant drift via PSI, and flags high collinearity using Pearson correlation.

Removing highly correlated features involves calculating Pearson correlation and identifying pairs with |r|≥0.7. The analysis generates a de-collinearity standard feature scheme that eliminates redundant variables, reducing multicollinearity risks before model training.

Yes, you can use pandas with parquet files for credit scoring feature analysis. The process requires a parquet dataset path, a binary target column, and optional exclude columns to generate a comprehensive Markdown report with recommended feature sets.

Generating a feature analysis report without model training involves running exploratory analysis on your full feature set. The process outputs a Markdown report containing descriptive statistics, IV rankings, PSI diagnostics, and four proposed modeling feature schemes for immediate selection planning.

A binary target column is required for feature selection planning because IV computation and predictive power evaluation depend on binary classification labels. Without a target variable present, the analysis cannot calculate feature relevance or generate the four recommended modeling feature schemes.