refactor:scikit-learn

Prevent data leakage in scikit-learn by encapsulating preprocessing, feature engineering, and modeling steps within a single Pipeline and ColumnTransformer, ensuring transformations are fitted only on training data during cross-validation.

Refactor machine learning code by moving fragmented scripts into a unified scikit-learn Pipeline, applying ColumnTransformer for heterogeneous data, enforcing deterministic random_state, and using cross-validation to ensure reproducible experiments.

Yes, scikit-learn ColumnTransformer works with custom transformers for feature engineering as long as they conform to the sklearn API, allowing you to handle heterogeneous data while maintaining pipeline compatibility and preventing data leakage.

The best way to ensure reproducibility across scikit-learn experiments is to set a deterministic random_state for all algorithms and encapsulate your workflow in a Pipeline with systematic hyperparameter tuning and proper cross-validation.

Scikit-learn cross-validation produces inconsistent results when random_state is not fixed or preprocessing steps are applied outside the Pipeline, causing data leakage. Encapsulate transformations in a Pipeline and set deterministic seeds.