Perform billion-scale k-NN similarity search with FAISS index types and GPU acceleration.
npx skills add https://github.com/zechenzhangAGI/AI-research-SKILLs --skill faiss
Please help me install this Agent Skill. Skill: faiss Source: https://github.com/zechenzhangAGI/AI-research-SKILLs/tree/main/15-rag/faiss Command: npx skills add https://github.com/zechenzhangAGI/AI-research-SKILLs --skill faiss
This Skill addresses the critical need for ultra-fast similarity searches on massive datasets of dense vectors, often involving millions or billions of items. It provides a high-performance solution for applications where speed and scale are paramount, without the overhead of a full database.
Create a NumPy array of 1000 random 128-dimensional vectors. Initialize a faiss.IndexFlatL2 index, add the vectors, then search for the 5 nearest neighbors to a query vector.
High-intent search queries and answers about installing and using this skill.
Similarity search on large vector datasets uses approximate nearest neighbor indexing to find similar items in milliseconds. FAISS provides multiple index types—Flat for exact search, IVF for approximate search on larger scales, and HNSW for balanced quality and speed—enabling billion-scale retrieval without exhaustive comparison.
Yes, GPU acceleration in vector search frameworks like FAISS achieves 10–100× faster query speeds by offloading index operations to GPUs. This drastically reduces latency for high-throughput nearest neighbor retrieval on massive datasets, making it practical for real-time recommendation and retrieval workloads.
Indexing billions of vectors requires choosing an appropriate index type based on accuracy and speed trade-offs: Flat indexes guarantee exact results but are slow; IVF indexes partition vectors into clusters for faster approximate search; HNSW indexes provide hierarchical navigation for quality-speed balance. Train the index on a representative sample, add vectors, then search using your query vector.
No, embedding retrieval at scale doesn't require a database if you only need pure similarity matching without metadata filtering. Specialized vector search libraries handle billion-scale k-NN retrieval efficiently in memory or on disk, reducing operational overhead while maintaining high throughput and low latency.
High-performance vector search libraries like FAISS support multiple index types optimized for different scenarios: Flat indexes for exact similarity, IVF for approximate search at scale, HNSW for hierarchical navigation, and Product Quantization for memory efficiency. Each trades off accuracy, speed, and memory consumption.
Yes, vector similarity search supports offline batch processing where embeddings are generated, indexed, and stored for later retrieval. This decouples expensive embedding computation from query time, enabling efficient preprocessing of large collections before serving similarity queries.
cardmagicClassify text sentiment and spam using pre-trained CLI models.
discountedcookieStore 384-dimensional embeddings in Postgres and query cosine similarity with pgvector.
qcmuuScale ML data preprocessing and batch inference with Ray Data across distributed workers.
Orchestra-ResearchPerform vector similarity and hybrid metadata searches with Qdrant.
