light-experiment-coding

Split data before fitting any preprocessor, then call fit or fit_transform only on the training fold and transform on the test fold. For cross-validation, place preprocessors and the model inside an sklearn Pipeline so each fold is fitted only on its training partition, and use group-aware splits so entities never cross train and test.

Set PYTHONHASHSEED before process start, then seed random, NumPy, torch.manual_seed, and torch.cuda.manual_seed_all, and enable cuDNN deterministic mode. Also pass an explicit generator and worker_init_fn to DataLoader, and record device, dtype, and known nondeterministic operators in the run manifest.

Write gold tests with hand-computable answers, property tests with Hypothesis for invariants like range and finiteness, metamorphic tests for output relations under input permutations, and train-only-fit tests proving transformers never see test data. Watch them fail first, then write the minimal implementation to pass.

No. Static AST and regex scanning only detects code-level patterns like fitting before splitting and can miss or misreport cases. The same applies to seed audits: matching two same-seed runs only supports same-environment reproducibility, not cross-hardware or cross-platform guarantees.

Failed runs preserve failure artifacts, exit codes, and stdout/stderr needed to classify root causes such as OOM, timeout, or code bugs. Deleting or overwriting them destroys auditability, and downstream result analysis must be able to recompute findings from complete raw evidence.