skills.rest
EngineerWithAI
EngineerWithAIOFFICIAL
Explore all skills by this author →
GitHub

embedding-strategies

Select and optimize embedding models and chunking strategies for RAG applications.

4|Updated Jan 10, 2026
Search Tags:#semantic search#embeddings#llm#rag#vector search#chunking
One-click install
npx skills add https://github.com/EngineerWithAI/engineerwith-agents --skill embedding-strategies-engineerwithai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: embedding-strategies
Source: https://github.com/EngineerWithAI/engineerwith-agents/tree/main/plugins/llm-application-dev/skills/embedding-strategies
Command: npx skills add https://github.com/EngineerWithAI/engineerwith-agents --skill embedding-strategies-engineerwithai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires tiktoken, nltk, sentence-transformers, numpy, openai, scipy, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps you choose and fine-tune the best embedding models and chunking strategies for your specific data, ensuring accurate semantic search and efficient Retrieval Augmented Generation (RAG) in your AI applications.

Core Features & Use Cases

  • Model Selection: Compare various embedding models (OpenAI, Sentence Transformers, Voyage) based on dimensions, cost, and best use cases (code, multilingual, general).
  • Chunking Strategies: Implement different methods like token-based, sentence-based, or recursive splitting to optimize context preservation.
  • Pipeline Implementation: Provides Python templates for both OpenAI and local embedding models, including preprocessing and embedding generation.
  • Quality Evaluation: Includes functions to evaluate retrieval quality using metrics like Precision@K, Recall@K, MRR, and NDCG.
  • Use Case: You are building a RAG system for legal documents. This skill will guide you in selecting an appropriate embedding model (e.g., voyage-2 for legal text), choosing a sentence-based chunking strategy to maintain legal context, and provides code to generate embeddings.

Quick Start

Use the embedding-strategies skill to generate embeddings for a list of documents using the text-embedding-3-small model.

Frequently Asked Questions about embedding-strategies

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose the best embedding model for RAG and semantic search?▼

To choose the best embedding model for RAG, compare OpenAI, Sentence Transformers, and Voyage options based on dimensions, cost, and specific use cases like multilingual or legal text. This selection process ensures accurate retrieval and efficient generation in your AI applications.

What is the best way to implement chunking strategies for vector search?▼

The best way to implement chunking strategies for vector search is by applying token-based, sentence-based, or recursive splitting methods. This optimization preserves document context and improves retrieval accuracy before generating your final embeddings.

How do I evaluate retrieval quality for semantic search applications?▼

You evaluate retrieval quality for semantic search using metrics like Precision@K, Recall@K, MRR, and NDCG. These evaluation functions measure how effectively your embedding models and chunking strategies retrieve relevant context from your vector database.

Can I use local embedding models instead of OpenAI for my RAG pipeline?▼

Yes, you can use local embedding models instead of OpenAI for your RAG pipeline. The skill provides Python templates for both local Sentence Transformers and OpenAI APIs, including necessary preprocessing and embedding generation steps for your documents.

Does chunking strategy affect semantic search accuracy for domain-specific text?▼

Chunking strategy directly affects semantic search accuracy for domain-specific text by preserving critical context. Using sentence-based chunking maintains legal context, while recursive splitting prevents breaking semantic meaning across arbitrary token limits within your documents.

Related Skills

Explore More Software Engineering Skills→
vecearvecear

embedding-pipeline-builder

Build document embedding and retrieval pipelines for scalable semantic search.

Community
Advanced
jayll1303jayll1303
17

text-embeddings-rag

Generate local embeddings and build retrieval-augmented generation pipelines with sentence-transformers and FAISS.

Community
Advanced
anukkrit149anukkrit149

rag-implementation

Build Retrieval-Augmented Generation systems grounding LLM answers in vector databases.

Community
Advanced
HemantSudarshanHemantSudarshan

rag-implementation

Implement production RAG pipelines from document ingestion to generation.

Community
Advanced