anti-cherry-pick

To prevent cherry-picking in experiment reporting, enforce mandatory documentation of all seeds, results, and failures with aggregated statistics like mean ± std or 95% CI. This approach ensures full transparency and reproducibility by requiring that no outcome is selectively omitted.

Documenting negative results in research requires storing them as data in a dedicated file like docs/05_execution/negative-results.md. This ensures reproducibility and auditability by guaranteeing that no failed experiment outcomes are silently dropped or deleted.

A fair baseline in machine learning experiments requires allocating equal compute budgets, data, preprocessing pipelines, and hyperparameter search budgets across all methods. This ensures equivalent access to shared pretrained models and external resources for unbiased evaluation.

You need to enforce complete reporting gates during Phase 4 execution and maintain them until project end. This discipline ensures full transparency across all datasets listed in the evaluation protocol, securing reproducibility and auditability throughout the lifecycle.

Yes, complete reporting requires equivalent access to ensure all methods share the same pretrained models, data, and external resources. This gate-based discipline prevents skewed conclusions by verifying fairness across all evaluated datasets.

Aggregated statistics such as mean ± std or 95% CI should be used instead of individual best results to avoid selective outcomes from skewing conclusions. This reporting discipline enforces full transparency by capturing the true variance across all seeds and failures.