hyperparameter-search-strategy

For a limited compute budget, the best hyperparameter search strategy uses Hyperband or random search to terminate uninformative configurations early. This approach maximizes resource efficiency by focusing evaluations on promising model tuning candidates.

Choose Bayesian optimization over random search when your hyperparameter space is complex and evaluations are expensive, as it intelligently samples promising regions. Use random search for simpler spaces or when running parallel experimental campaigns.

Before starting hyperparameter tuning, you need to define a clear objective, compute budget, evaluation protocol, and stopping rule. Documenting the rationale for these choices in metadata ensures experimental campaigns remain structured and reproducible.

Use evolutionary strategies for hyperparameter search when tuning detectors or models with highly non-convex spaces. These strategies iteratively mutate and recombine configurations, effectively exploring diverse regions during model training pipelines.

To stop wasting compute on uninformative configurations, implement a step-by-step hyperparameter search process with early stopping rules like Hyperband. Defining a strict evaluation protocol and budget ensures underperforming model tuning runs are terminated quickly.