skills.rest
rajyeole6
rajyeole6COMMUNITY
Explore all skills by this author →
GitHub

agency-search-relevance-engineer

Design hybrid BM25 and vector search infrastructure for Elasticsearch and OpenSearch.

Updated Jul 23, 2026
Search Tags:#search#relevance#elasticsearch#opensearch#hybrid-search#ndcg
One-click install
npx skills add https://github.com/rajyeole6/AI-RECRUITER --skill agency-search-relevance-engineer-rajyeole6
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agency-search-relevance-engineer
Source: https://github.com/rajyeole6/AI-RECRUITER/tree/main/.agents/skills/engineering-search-relevance-engineer
Command: npx skills add https://github.com/rajyeole6/AI-RECRUITER --skill agency-search-relevance-engineer-rajyeole6

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill addresses the common failure of search systems to provide accurate, relevant results by replacing anecdotal tuning with a rigorous, metrics-driven engineering discipline.

Core Features & Use Cases

  • Relevance Evaluation: Implements nDCG and MRR scoring against golden judgment sets to validate changes before deployment.
  • Hybrid Retrieval: Orchestrates BM25 lexical search with vector-based semantic retrieval using Reciprocal Rank Fusion (RRF).
  • Infrastructure Design: Provides blueprints for analyzer chains, index aliasing, and zero-downtime reindexing strategies.

Quick Start

Use the agency-search-relevance-engineer skill to evaluate the current search index against the provided judgment set and suggest improvements for the tail query performance.

Frequently Asked Questions about agency-search-relevance-engineer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I balance lexical BM25 retrieval with semantic vector similarity in Elasticsearch?▼

Hybrid search balances BM25 and vector similarity by orchestrating both retrieval methods using Reciprocal Rank Fusion (RRF). This approach merges lexical and semantic results to achieve high recall and precision in production search systems.

How do I evaluate search relevance changes against judgment sets before deployment?▼

Search relevance evaluation uses nDCG and MRR scoring against golden judgment sets to rigorously validate changes offline. This metrics-driven discipline replaces anecdotal tuning and ensures improvements before any production deployment.

What is the best way to update Elasticsearch index mappings without downtime?▼

Zero-downtime reindexing requires infrastructure design utilizing index aliasing and analyzer chain blueprints. This strategy allows you to update mappings and analyzers in production while maintaining continuous search availability.

Can I use this approach to improve tail query performance in OpenSearch?▼

Yes, optimizing tail query performance in OpenSearch involves evaluating the current search index against judgment sets. The skill identifies tail query issues and suggests targeted improvements for low-frequency searches.

Does hybrid search with Reciprocal Rank Fusion require low-latency infrastructure?▼

Production-grade hybrid search using RRF requires infrastructure designed for low-latency performance. Balancing BM25 and vector retrieval efficiently ensures the system maintains fast response times while delivering relevant results.

Related Skills

Explore More Software Engineering Skills→
imMamdouhaboammarimMamdouhaboammar

agency-search-relevance-engineer

Design Elasticsearch and OpenSearch relevance tuning with nDCG evaluation.

Community
Advanced
hamelsmuhamelsmu
1.6k

evaluate-rag

Evaluate RAG pipelines by measuring retrieval metrics and generation quality.

Community
Advanced
NotysotyNotysoty
9

Hybrid Search Architect

Design hybrid retrieval pipelines combining dense vector search with BM25 keyword matching.

Community
Advanced
thistleknotthistleknot

gist-retriever

Fuse BM25 and GIST seeds with RRF and ColBERT reranking for GraphRAG retrieval.

Community
Advanced