rag-query-debug

Diagnose RAG queries returning wrong, low-quality, or empty retrieval results.

Updated Nov 19, 2020
One-click install
npx skills add https://github.com/kwojtasinski-repo/ECommerceApp --skill rag-query-debug-kwojtasinski-repo
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: rag-query-debug
Source: https://github.com/kwojtasinski-repo/ECommerceApp/tree/main/.github/skills/rag-query-debug
Command: npx skills add https://github.com/kwojtasinski-repo/ECommerceApp --skill rag-query-debug-kwojtasinski-repo

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? A RAG query returns the wrong file at top-1, uniformly low scores, or empty results, and you need a systematic way to find the root cause instead of guessing at weight tweaks or re-indexing. ## Core Features & Use Cases - Hypothesis-ordered checklist: Walks chunk presence, score gaps, weight policy, multilingual expansion, and server asymmetry in a fixed order, stopping at the first concrete finding. - Concrete probe commands: Provides ready-to-run Qdrant scroll queries, probe_weights.py invocations, and compare_queries.py parity audits for both Python and .NET retrieval servers. - Guardrails against dead ends: Documents weight bounds (0.80-1.30), when --force-full re-indexing is actually required, and which fixes (embedder swaps, blind re-ingestion) to avoid. - Use Case: A user asks why the query "payment refund policy" returns an ADR instead of the payments lifecycle spec; the skill guides checking corpus presence, comparing raw versus post-weight scores, and confirming whether an over-boosted weight caused the mismatch. ## Quick Start Debug why the RAG query "how are inventory reservations released" returns the wrong document instead of the inventory spec.

Frequently Asked Questions about rag-query-debug

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I debug a RAG query that returns the wrong document?

First define the expected top-1 result, then verify the file exists in the Qdrant collection, and run probe_weights.py to compare raw versus post-weight scores on both servers. The checklist isolates whether the cause is missing ingestion, weight boosting, chunking, or language mismatch.

Why does my RAG query return empty or low-score results?

Scores below 0.25 on both servers usually mean the embedder does not understand the query, often due to language mismatch with the corpus. Rephrase the query, add synonyms, or expand the multilingual glossary before changing any weights.

How do I check if a file was ingested into Qdrant?

Run a Qdrant scroll query filtered on the rel_path payload field against the collection. No hits means the file was never ingested, so check metadata-rules.yaml exclude patterns or run the ingest script.

When should I re-index the RAG collection?

Re-index with --force-full only after changing chunker settings or metadata rules, since content hashes prevent re-embedding otherwise. Re-running ingestion never fixes a bad query on its own.

What are safe weight limits for RAG retrieval tuning?

Keep weights between 0.80 and 1.30. Raising weights above 1.30 destabilizes ranking on unrelated queries, and lowering below 0.80 hides genuinely relevant content.

Why do Python and .NET RAG servers return different results?

Different winners across servers indicate asymmetric retrieval, not a weight problem. Common causes include over-boosting of decision documents on past-tense queries and chunk-level doc_kind misclassification; run compare_queries.py for a parity audit.