skills.rest
jayll1303
jayll1303COMMUNITY
Explore all skills by this author →
GitHub

text-embeddings-rag

Generate local embeddings and build retrieval-augmented generation pipelines with sentence-transformers and FAISS.

17|3|Updated Apr 3, 2026
Search Tags:#embeddings#rag#qdrant#semantic-search#sentence-transformers#faiss#chromadb
One-click install
npx skills add https://github.com/jayll1303/AIEKit --skill text-embeddings-rag
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: text-embeddings-rag
Source: https://github.com/jayll1303/AIEKit/tree/main/.kiro/skills/text-embeddings-rag
Command: npx skills add https://github.com/jayll1303/AIEKit --skill text-embeddings-rag

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Turn unstructured document collections into dense embeddings and reliable retrieval context so that language models can answer questions accurately without brittle keyword searches or manual document inspection. The skill consolidates guidance on embedding generation, vector indexing, retrieval tuning, and integration with LLM inference to make RAG pipelines reproducible and maintainable.

Core Features & Use Cases

  • Local embedding generation using sentence-transformers with GPU-accelerated batch encoding and L2 normalization for cosine search.
  • Vector store integration patterns and validation for FAISS, ChromaDB, and Qdrant including persistence, ID mapping, and payload/metadata handling.
  • End-to-end RAG pipelines: document chunking, embedding, indexing, retrieval, and generation with examples for retrieval + re-ranking and hybrid BM25+vector search.
  • Retrieval optimization techniques: chunk size tuning, overlap strategies, cross-encoder re-ranking, reciprocal rank fusion, and metadata filtering.
  • Language-specific guidance such as Vietnamese VN-MTEB recommendations and word-segmentation advice for BM25 hybrid search.
  • Use Case: Index internal product docs, retrieve top context for a user question, and synthesize a grounded answer with source attributions.

Quick Start

Use the text-embeddings-rag skill to index your document corpus with sentence-transformers, persist vectors in FAISS or ChromaDB, and run a retrieval-augmented query to generate a context-aware answer.

Frequently Asked Questions about text-embeddings-rag

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a local RAG pipeline for semantic search using sentence-transformers?▼

Build a local RAG pipeline by generating L2-normalized embeddings with sentence-transformers, indexing vectors in FAISS or ChromaDB, and retrieving top context for grounded LLM generation.

What is the best way to persist vector embeddings and filter metadata in Qdrant?▼

Qdrant vector store integration supports persistence, ID mapping, and payload handling to enable metadata filtering during semantic search retrieval.

How does hybrid BM25 plus vector search improve retrieval accuracy?▼

Hybrid BM25 plus vector search combines sparse keyword matching with dense embeddings, applying reciprocal rank fusion and cross-encoder re-ranking to optimize retrieval accuracy.

Can I use GPU-accelerated batch encoding for local embedding generation?▼

Local embedding generation supports GPU-accelerated batch encoding with sentence-transformers to produce L2-normalized vectors optimized for cosine similarity search.

Does this RAG pipeline approach support language-specific tokenization like Vietnamese word-segmentation?▼

Vietnamese word-segmentation and VN-MTEB recommendations are supported to optimize BM25 hybrid search tokenization and embedding generation for language-specific contexts.

Related Skills

Explore More Software Engineering Skills→
giuseppe-trisciuogliogiuseppe-trisciuoglio
322

chunking-strategy

Chunk documents into semantically meaningful segments for RAG systems.

Community
Advanced
LauraFlorentinLauraFlorentin
1

rag

Retrieves relevant documents from a knowledge base to ground responses.

Community
Advanced
NotysotyNotysoty
9

Hybrid Search Architect

Design hybrid retrieval pipelines combining dense vector search with BM25 keyword matching.

Community
Advanced
vertivolatamvertivolatam
1

OVHCloud Backend Deployment

Deploy Flutter backends on OVHCloud with managed Kubernetes and PostgreSQL.

Official
Intermediate