ml-science-discipline

Prevent data leakage by enforcing leakage-resistant preprocessing that fits transformations only on training data. This ensures preprocessing happens before splitting or strictly within cross-validation folds, keeping the test set sealed and preventing invalid model evaluation.

The best way to design cross-validation for tuning is using nested cross-validation. This separates hyperparameter tuning from model evaluation, ensuring the test set remains sealed and providing multi-seed reporting variance for publication-grade statistical testing.

Report machine learning results for peer review by generating compliance-oriented artifacts aligned with AI reporting standards. Include statistical comparisons with effect sizes, calibration and uncertainty metrics, and multi-seed variance rather than single lucky runs.

You need stratified splitting to maintain class proportions, group splitting to prevent patient or cluster leakage, and temporal splitting for time-series data. Choosing the correct split strategy ensures valid decision-relevant claims and a sealed test set.

Plan external validation for distribution shift by designing experiments that test models against out-of-distribution data. This involves selecting appropriate metrics and running statistical testing to verify model performance and calibration under realistic deployment conditions.

Pre-register hypotheses and baselines to enforce reproducible, hypothesis-first experimentation. Defining falsifiable hypotheses and your primary metric before training prevents p-hacking and ensures publication-standard, decision-relevant claims.