Run resumable Optuna hyperparameter searches with nested cross-validation and SQLite persistence.
npx skills add https://github.com/thistleknot/skills --skill optuna-nested-cv
Please help me install this Agent Skill. Skill: optuna-nested-cv Source: https://github.com/thistleknot/skills/tree/main/optuna-nested-cv Command: npx skills add https://github.com/thistleknot/skills --skill optuna-nested-cv
It eliminates unreliable hyperparameter tuning by running a persistent Optuna search inside a nested cross-validation structure, producing an unbiased generalization estimate while preventing holdout leakage.
Use the optuna-nested-cv skill to run an Optuna TPE hyperparameter search over an inner loop for each outer CV fold, persisting trials to sqlite and reporting the mean outer-fold score as the unbiased estimate.
High-intent search queries and answers about installing and using this skill.
Resumable Optuna hyperparameter tuning with nested cross-validation runs a TPE search inside an inner loop for each outer fold, persisting trials to SQLite to ensure the process is auditable and restartable.
Hyperparameter optimization leaks holdout data when the tuning process evaluates on the same split used for final testing; nested cross-validation prevents this by enforcing strict tune versus holdout separation across outer folds.
Yes, TPE sampler searches in Optuna support a single scalar objective, which can be a composite metric, ensuring the model selection process remains unambiguous during the hyperparameter optimization.
Optuna supports SQLite-backed study persistence for expensive ML pipelines by using a stable study name and persistent storage, allowing trials to be saved, audited, and restarted across search sessions.
The best way to get an unbiased generalization score from hyperparameter tuning is nested cross-validation, which evaluates the model on an outer holdout fold after tuning only on the inner training split.
Fold-safe preprocessing normalization during Optuna model selection requires applying normalization independently within each split, ensuring preprocessing parameters are fitted only on the inner training data to prevent data leakage.
thistleknotStructures hyperparameter tuning as experimental design with layerwise optimization and trial logging.
aliyunOptimize XGBoost hyperparameters via diagnostic-driven Optuna TPE search.
CC90210Store cross-session tool usage and file edits in SQLite with semantic search.
KameniAlexNeaRun Bayesian hyperparameter optimization for LightGBM, XGBoost, and CatBoost models with Optuna.
