skills.rest
SanketAdlak
SanketAdlakCOMMUNITY
Explore all skills by this author →
Website𝕏@devsanket14GitHub

vector-index-tuning

Optimize HNSW parameters, quantization strategies, and memory usage for vector search indexes.

Updated Apr 23, 2026
Search Tags:#embeddings#vector search#qdrant#quantization#hnsw#index tuning#recall optimization
One-click install
npx skills add https://github.com/SanketAdlak/PDMProjectDesign --skill vector-index-tuning-sanketadlak
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: vector-index-tuning
Source: https://github.com/SanketAdlak/PDMProjectDesign/tree/main/.agents/skills/vector-index-tuning
Command: npx skills add https://github.com/SanketAdlak/PDMProjectDesign --skill vector-index-tuning-sanketadlak

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires numpy, hnswlib, scikit-learn, qdrant-client.

What problem does it solve? Vector search systems often suffer from high latency, poor recall, or excessive memory consumption when indexes are left at default settings. This Skill provides concrete guidance and code templates for tuning vector indexes to meet production latency, recall, and memory targets. ## Core Features & Use Cases - HNSW Parameter Benchmarking: Systematically benchmark M, efConstruction, and efSearch values against ground truth to find optimal recall-latency tradeoffs. - Quantization Strategies: Implement INT8 scalar, product, and binary quantization to compress vector storage by 4x-32x. - Qdrant Configuration: Create collections pre-configured for recall, speed, balanced, or memory optimization targets. - Use Case: You have 10 million embeddings and search latency exceeds your 10ms budget. Use the benchmarking template to sweep HNSW parameters, apply INT8 quantization to cut memory by 4x, and validate recall stays above 95%. ## Quick Start Ask the AI to recommend HNSW parameters and a quantization strategy for your dataset size, recall target, and latency budget.

Frequently Asked Questions about vector-index-tuning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I tune HNSW parameters for better recall?▼

Increase efSearch first since it directly controls search-time recall without rebuilding the index. If recall is still insufficient, raise M and efConstruction and rebuild the index, accepting higher memory usage and slower build times.

What quantization method should I use for vector compression?▼

INT8 scalar quantization offers a 4x reduction with minimal recall loss and works well for most cases. Product quantization achieves roughly 16x compression for very large datasets, while binary quantization gives 32x compression but suits only high-dimensional embeddings.

Which vector index type works best for my dataset size?▼

Use flat exact search under 10K vectors, HNSW for 10K to 1M, HNSW with quantization for 1M to 100M, and IVF with product quantization or DiskANN beyond 100M vectors.

How do I configure Qdrant for low memory usage?▼

Create the collection with product quantization at 16x compression, set HNSW M to 8, and lower the memmap threshold so segments move to disk sooner. This trades some recall and latency for significantly reduced RAM consumption.

Why is my vector search recall lower than expected?▼

Low recall usually comes from efSearch being too small, quantization error without rescoring, or an index built with low efConstruction. Enable rescoring with oversampling for quantized searches and benchmark against brute-force ground truth to isolate the cause.

When should I not use HNSW indexing?▼

Avoid HNSW for datasets under 10K vectors where exact flat search is fast and perfectly accurate, or when memory is severely constrained since the graph structure adds significant per-vector overhead.

Related Skills

Explore More Software Engineering Skills→
lewisperez999lewisperez999

vector-embeddings

Generate text embeddings with OpenAI and search by cosine similarity in Upstash Vector or pg-vector.

Community
Intermediate
MrJmpl3MrJmpl3

0170-similarity-search-patterns

Select distance metrics and vector index structures for nearest-neighbor retrieval.

Community
Intermediate
LightricksLightricks

api-runtime-monitor

Monitor LTX API latency, error rates, and throughput with threshold alerts.

Official
Advanced
zerwizzerwiz

faiss

Optimizes FAISS index selection and GPU-accelerated similarity search for dense embeddings.

Community
Intermediate