optimize-cortex-search-service

You optimize Cortex Search ranking weights by applying Bayesian optimization across text, vector, and optional reranker weights to maximize ndcg and recall for a suite of queries. This Skill automates the entire workflow from query sampling to final weight configuration.

Yes, you can use LLM-assisted scoring to evaluate unjudged documents during search optimization. The process includes caching these LLM scores to accelerate subsequent optimization runs and reduce redundant API calls.

The best way to tune vector and text weights is using Bayesian optimization with Optuna to systematically explore weight combinations. It evaluates retrieval quality using ndcg and recall metrics to produce the best-weight configuration for your service.

No, you do not strictly need a prepared queries file to run Cortex Search optimization. The workflow supports generating a query set automatically, allowing you to execute end-to-end optimization even without pre-existing query data.

Bayesian optimization caches unjudged document scores generated by the LLM during evaluation. This caching mechanism prevents redundant scoring computations, significantly accelerating subsequent optimization runs across multiple query sets.

A limitation of LLM-assisted scoring is the potential latency and API cost overhead during the initial evaluation of unjudged documents. However, built-in caching mitigates this for subsequent runs by storing previously computed scores.