skills.rest
wshobson
wshobsonCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

vector-index-tuning

Tune HNSW parameters and quantization strategies for vector index performance.

38.6k|4.1k|Updated Jul 24, 2025
Search Tags:#vector database#performance tuning#similarity search#quantization#Qdrant#HNSW
One-click install
npx skills add https://github.com/wshobson/agents --skill vector-index-tuning-wshobson
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: vector-index-tuning
Source: https://github.com/wshobson/agents/tree/main/plugins/llm-application-dev/skills/vector-index-tuning
Command: npx skills add https://github.com/wshobson/agents --skill vector-index-tuning-wshobson

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires hnswlib, sklearn, qdrant-client, numpy, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of optimizing vector index performance, ensuring efficient and accurate search results by fine-tuning parameters related to latency, recall, and memory usage.

Core Features & Use Cases

  • HNSW Parameter Tuning: Adjust M, efConstruction, and efSearch for optimal recall and speed.
  • Quantization Strategies: Implement INT8 scalar, Product Quantization (PQ), or binary quantization to reduce memory footprint.
  • Index Type Selection: Guidance on choosing between Flat, HNSW, IVF, and DiskANN based on data size.
  • Performance Monitoring: Tools to benchmark build times, search latency, and recall.
  • Use Case: When deploying a large-scale similarity search system, use this Skill to select the best HNSW parameters and quantization method to meet strict latency requirements while maintaining high search accuracy.

Quick Start

Use the vector-index-tuning skill to recommend HNSW parameters for a dataset of 500,000 vectors targeting 95% recall.

Frequently Asked Questions about vector-index-tuning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I tune HNSW parameters for optimal vector search recall and latency?▼

HNSW parameter tuning adjusts M, efConstruction, and efSearch to balance vector search recall and speed. This Skill provides templates to configure these parameters for large-scale similarity search systems targeting specific accuracy thresholds.

What's the best way to reduce vector database memory usage without losing search accuracy?▼

Reducing vector database memory footprint uses quantization strategies like INT8 scalar, Product Quantization (PQ), or binary quantization. This Skill implements these techniques to maintain high search accuracy while significantly decreasing memory consumption.

How do I choose between Flat, HNSW, IVF, and DiskANN vector indexes for my dataset?▼

Choosing the right vector index type depends on dataset size and latency requirements. This Skill provides guidance on selecting among Flat, HNSW, IVF, and DiskANN indexes to ensure optimal similarity search performance for your data scale.

Does Qdrant support HNSW tuning and quantization for large-scale similarity search?▼

Qdrant supports HNSW tuning and quantization for large-scale similarity search. This Skill provides Qdrant configuration templates to optimize search latency, recall, and memory efficiency using these vector index performance techniques.

How do I benchmark vector search performance and monitor recall improvements?▼

Benchmarking vector search performance monitors build times, search latency, and recall metrics. This Skill provides performance monitoring tools to validate HNSW tuning and quantization strategies against strict latency requirements.

When should I use Product Quantization over scalar quantization in my vector database?▼

Product Quantization compresses high-dimensional vectors more aggressively than INT8 scalar quantization, yielding greater memory savings. This Skill helps select the appropriate quantization strategy based on your latency and recall constraints.

Related Skills

Explore More Software Engineering Skills→
ccfccf

embedding-strategies

Select and optimize embedding models for semantic search and RAG applications.

Community
Intermediate
EngineerWithAIEngineerWithAI
4

vector-index-tuning

Tune HNSW parameters and quantization strategies for Qdrant vector indexes.

Official
Advanced
ccfccf

vector-index-tuning

Tune HNSW parameters and quantization for large-scale vector search indexes.

Community
Advanced
DrLuggelsDrLuggels
3

vector-index-tuning

Optimize vector index performance for latency, recall, and memory usage.

Community
Advanced