tune-rag-weights

Adjusts ranking weight multipliers in a RAG configuration file to reorder query results.

Updated Nov 19, 2020
One-click install
npx skills add https://github.com/kwojtasinski-repo/ECommerceApp --skill tune-rag-weights-kwojtasinski-repo
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: tune-rag-weights
Source: https://github.com/kwojtasinski-repo/ECommerceApp/tree/main/.github/skills/tune-rag-weights
Command: npx skills add https://github.com/kwojtasinski-repo/ECommerceApp --skill tune-rag-weights-kwojtasinski-repo

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When a retrieval-augmented generation system returns the wrong files at the top of query results despite good semantic similarity, this Skill fixes the ranking by editing weight multipliers in the RAG config — no re-indexing or re-embedding required. ## Core Features & Use Cases - Query-time weight tuning: Edits the ranking.weights table in tools/rag/rag-config.yaml using fnmatch glob patterns, with first-match-wins ordering. - Guided diagnosis workflow: Runs failing queries, interprets score gaps, and maps symptoms (right file ranked low, wrong file winning, missing file) to specific weight adjustments. - Regression validation: Re-runs the query immediately and executes the full eval benchmark (eval.py) to confirm no previously passing queries regressed. - Use Case: An ADR about coupons consistently ranks below router README pages for coupon queries. Add a glob rule docs/adr/0016/** with weight 1.15 before the broader patterns, then verify the file moves to position one. ## Quick Start Ask the assistant to tune the RAG ranking weights so that a specific file or folder ranks higher or lower for a given query, providing the file path and whether to boost or suppress it.

Frequently Asked Questions about tune-rag-weights

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I boost a file's ranking in RAG query results?

Add a glob pattern for the file to the ranking.weights table in rag-config.yaml with a multiplier above 1.00, such as 1.15. Place more specific patterns before broader globs because the first match wins, then re-run the query to confirm the new position.

How to suppress a document that always ranks first in vector search?

Lower its weight multiplier below 1.00 in the ranking.weights config, typically to 0.85–0.95 for mild suppression or 0.70–0.89 for strong suppression. Weights apply at query time, so no re-indexing is needed.

Do RAG weight changes require re-indexing embeddings?

No. Weight multipliers are applied at query time on top of the raw cosine similarity score from Qdrant, so changes take effect immediately after saving the config file.

Why does the right file never appear in RAG query results?

If the file never appears at all, the cause is likely a language or vocabulary gap rather than a weight problem, so check the glossary first. Weights only reorder files that already receive a similarity score.

What is the maximum safe weight multiplier for RAG ranking?

Do not set weights above 1.40, because such values dominate every query regardless of semantic fit. Reserve 1.30 or higher only for files that must always be consulted first, such as known-issues lists.