vector-index-tuning

Tune HNSW parameters and quantization strategies for vector index performance.

38.6k|4.1k|Updated Jul 24, 2025

Search Tags:#vector database #performance tuning #similarity search #quantization #Qdrant #HNSW

Ready to run

npx skills add https://github.com/wshobson/agents --skill vector-index-tuning-wshobson

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: vector-index-tuning
Source: https://github.com/wshobson/agents/tree/main/plugins/llm-application-dev/skills/vector-index-tuning
Command: npx skills add https://github.com/wshobson/agents --skill vector-index-tuning-wshobson

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires hnswlib, sklearn, qdrant-client, numpy, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of optimizing vector index performance, ensuring efficient and accurate search results by fine-tuning parameters related to latency, recall, and memory usage.

Core Features & Use Cases

HNSW Parameter Tuning: Adjust M, efConstruction, and efSearch for optimal recall and speed.
Quantization Strategies: Implement INT8 scalar, Product Quantization (PQ), or binary quantization to reduce memory footprint.
Index Type Selection: Guidance on choosing between Flat, HNSW, IVF, and DiskANN based on data size.
Performance Monitoring: Tools to benchmark build times, search latency, and recall.
Use Case: When deploying a large-scale similarity search system, use this Skill to select the best HNSW parameters and quantization method to meet strict latency requirements while maintaining high search accuracy.

Quick Start

Use the vector-index-tuning skill to recommend HNSW parameters for a dataset of 500,000 vectors targeting 95% recall.

Frequently Asked Questions about vector-index-tuning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I tune HNSW parameters for optimal vector search recall and latency?▼

HNSW parameter tuning adjusts M, efConstruction, and efSearch to balance vector search recall and speed. This Skill provides templates to configure these parameters for large-scale similarity search systems targeting specific accuracy thresholds.

What's the best way to reduce vector database memory usage without losing search accuracy?▼

Reducing vector database memory footprint uses quantization strategies like INT8 scalar, Product Quantization (PQ), or binary quantization. This Skill implements these techniques to maintain high search accuracy while significantly decreasing memory consumption.

How do I choose between Flat, HNSW, IVF, and DiskANN vector indexes for my dataset?▼

Choosing the right vector index type depends on dataset size and latency requirements. This Skill provides guidance on selecting among Flat, HNSW, IVF, and DiskANN indexes to ensure optimal similarity search performance for your data scale.

Does Qdrant support HNSW tuning and quantization for large-scale similarity search?▼

Qdrant supports HNSW tuning and quantization for large-scale similarity search. This Skill provides Qdrant configuration templates to optimize search latency, recall, and memory efficiency using these vector index performance techniques.

How do I benchmark vector search performance and monitor recall improvements?▼

Benchmarking vector search performance monitors build times, search latency, and recall metrics. This Skill provides performance monitoring tools to validate HNSW tuning and quantization strategies against strict latency requirements.

When should I use Product Quantization over scalar quantization in my vector database?▼

Product Quantization compresses high-dimensional vectors more aggressively than INT8 scalar quantization, yielding greater memory savings. This Skill helps select the appropriate quantization strategy based on your latency and recall constraints.