feature-analysis

Compute descriptive statistics, IV, PSI stability, and Pearson correlation for binary finance datasets.

580|66|Updated Apr 21, 2025

Search Tags:#feature analysis #feature selection #credit scoring #psi stability #iv computation #correlation screening #modeling prep

Ready to run

npx skills add https://github.com/aliyun/qwen-dianjin --skill feature-analysis

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: feature-analysis
Source: https://github.com/aliyun/qwen-dianjin/tree/main/DianJin-SKILLS/financial-engineering-expert/feature-analysis
Command: npx skills add https://github.com/aliyun/qwen-dianjin --skill feature-analysis

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pandas, optbinning, numpy, scipy, and includes scripts (resource) components.

What problem does it solve?

This Skill resolves the problem of unclear feature quality in finance modeling by deeply analyzing each feature’s predictive power, stability over time, and redundancy risk so you can choose a reliable modeling feature set.

Core Features & Use Cases

IV值预测能力评估：对全量特征计算 IV 并给出 Top 排名，判断哪些特征可为建模带来区分度。
PSI稳定性诊断：在提供基准集/对比集切分条件（或自动切分条件满足）时，输出 PSI 并识别显著漂移特征。
相关性与共线性建议：计算 Pearson 相关性，识别 |r|≥0.7 的高相关特征对，并在方案中给出去共线性策略。
四套建模特征方案输出：生成全量入模、去共线性标准、高预测力精选、稳定性优先四套特征集合建议。

Quick Start

Ask the agent to run feature analysis on your parquet dataset by specifying the target label, exclude columns, and (optionally) baseline/comparison filters for PSI, then generate the feature_analysis_report.md report with the recommended feature sets.

Frequently Asked Questions about feature-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I perform feature selection for a binary classification dataset in finance?▼

Feature selection for a binary finance dataset requires analyzing predictive power, stability, and redundancy. This process computes IV, PSI stability, and Pearson correlation across all features to identify weak variables and output four distinct modeling feature schemes.

How do I calculate IV and PSI to evaluate feature stability and predictive power?▼

Calculating IV and PSI involves splitting your dataset into baseline and comparison segments to measure predictive value and population stability. The analysis ranks features by IV, identifies significant drift via PSI, and flags high collinearity using Pearson correlation.

How do I remove highly correlated features and handle multicollinearity before modeling?▼

Removing highly correlated features involves calculating Pearson correlation and identifying pairs with |r|≥0.7. The analysis generates a de-collinearity standard feature scheme that eliminates redundant variables, reducing multicollinearity risks before model training.

Can I use pandas and parquet files for credit scoring feature analysis?▼

Yes, you can use pandas with parquet files for credit scoring feature analysis. The process requires a parquet dataset path, a binary target column, and optional exclude columns to generate a comprehensive Markdown report with recommended feature sets.

What is the best way to generate a feature analysis report without training a model?▼

Generating a feature analysis report without model training involves running exploratory analysis on your full feature set. The process outputs a Markdown report containing descriptive statistics, IV rankings, PSI diagnostics, and four proposed modeling feature schemes for immediate selection planning.

Why do I need a binary target column for feature selection planning?▼

A binary target column is required for feature selection planning because IV computation and predictive power evaluation depend on binary classification labels. Without a target variable present, the analysis cannot calculate feature relevance or generate the four recommended modeling feature schemes.