rag-retrieval

Implements hybrid BM25 and vector retrieval with RRF fusion, reranking, and parent-chunk resolution.

Updated Aug 17, 2026
One-click install
npx skills add https://github.com/lakshya4568/DeepContext --skill rag-retrieval-lakshya4568
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: rag-retrieval
Source: https://github.com/lakshya4568/DeepContext/tree/main/.agents/skills/rag-retrieval
Command: npx skills add https://github.com/lakshya4568/DeepContext --skill rag-retrieval-lakshya4568

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Answering questions from ingested documents requires more than a single vector search: naive top-k retrieval loses recall, dilutes generation with raw child chunks, and has no way to signal insufficient evidence. This Skill implements the full hybrid retrieval pipeline so queries against a document corpus return ranked, citation-backed parent chunks or an explicit insufficiency signal. ## Core Features & Use Cases - Hybrid first-stage recall: Runs BM25 full-text search and pgvector dense vector search in parallel, then fuses results with Reciprocal Rank Fusion (k=60). - Cross-encoder reranking and parent resolution: Reranks the top 50-100 fused candidates down to 5-10, then resolves child chunks to their 1,000-2,500 token parent chunks before generation. - Permission-safe filtering and sufficiency gating: Compiles tenant_id and permission_scope into SQL WHERE clauses, and applies a bounded corrective-retry loop (max 1) before escalating aggregation queries to the RLM path. - Use Case: A user asks a multi-part question over a 1,000-page ingested document set; the pipeline classifies the query, decomposes it, retrieves and reranks evidence, and returns cited parent chunks or an "insufficient evidence" answer instead of a guess. ## Quick Start Use the rag-retrieval skill to answer this question from the ingested documents and return ranked parent chunks with citations.

Frequently Asked Questions about rag-retrieval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I implement hybrid search with BM25 and vector retrieval?

Run BM25 full-text search and dense vector search in parallel, each returning the top 50-100 child chunks, then fuse the ranked lists with Reciprocal Rank Fusion using k=60. Rerank the fused candidates with a cross-encoder and keep the top 5-10.

What is Reciprocal Rank Fusion and how does it work?

Reciprocal Rank Fusion combines multiple ranked lists by scoring each item as the sum of 1/(k + rank) across every list it appears in, with k around 60 as the standard constant. It dampens rank-1 dominance from any single retrieval leg.

Why should child chunks be resolved to parent chunks before generation?

The small child chunk (300-600 tokens) is only what was searched for precise matching; the larger parent chunk (1,000-2,500 tokens) carries the surrounding context the generator needs. Sending raw child text to the model loses macro context and degrades answers.

When should I switch from pgvector to Qdrant or Milvus?

Switch only when P95 vector-search latency exceeds budget after index tuning, when the corpus reaches tens of millions of vectors with high write throughput, or when multi-region replication is required. The retrieve() interface stays unchanged so callers are unaffected.

How do I enforce document permissions in RAG retrieval?

Compile tenant_id and permission_scope directly into SQL WHERE clauses against the chunks and documents tables, never as prompt instructions. A permission filter expressed only in prompt text is a data leak risk, not a UX issue.

What happens when retrieval returns insufficient evidence?

The pipeline rewrites the query and retries exactly once, then either escalates aggregation-style queries to the RLM recursion path or returns an explicit insufficient-evidence answer. Unbounded retry loops are explicitly prohibited.