generate-rag-rules

Update RAG document classification rules and named eval queries in YAML configs.

Updated Nov 19, 2020
One-click install
npx skills add https://github.com/kwojtasinski-repo/ECommerceApp --skill generate-rag-rules-kwojtasinski-repo
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: generate-rag-rules
Source: https://github.com/kwojtasinski-repo/ECommerceApp/tree/main/.github/skills/generate-rag-rules
Command: npx skills add https://github.com/kwojtasinski-repo/ECommerceApp --skill generate-rag-rules-kwojtasinski-repo

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When new documentation folders, file types, or ADR naming conventions are added to a repository, the RAG index misclassifies documents and eval queries lose coverage. This Skill guides correct updates to metadata-rules.yaml and queries.yaml so documents get the right doc_kind and retrieval quality stays measurable. ## Core Features & Use Cases - Document classification rules: Add or reorder fnmatch glob rules in metadata-rules.yaml so new folders and file types receive the correct doc_kind, with specific patterns placed before broad catch-alls. - Named eval queries: Extend queries.yaml with well-formed natural-language questions, optional doc_kind and adr_id filters, and top_k overrides for eval.py coverage tracking. - Re-index guidance: Distinguish changes requiring a force-full re-index (metadata-rules) from those that do not (queries.yaml), and verify results with eval.py or query_docs. - Use Case: After adding a docs/runbooks/ folder, use this Skill to add a runbook doc_kind rule, create a coverage query for it, run a force-full re-index, and confirm retrieval with eval.py. ## Quick Start Ask the agent to add a new doc_kind rule for the docs/runbooks folder and a matching coverage query in queries.yaml, then run eval.py to verify.

Frequently Asked Questions about generate-rag-rules

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new document kind to RAG metadata rules?

Add a fnmatch glob rule to metadata-rules.yaml assigning the new kind, placing it before any broad catch-all pattern that would match first. Then run a force-full re-index so existing chunks receive the new doc_kind.

How do I add a named eval query to queries.yaml?

Add an entry with a snake_case name, a natural-language question using exact terms from the target document, and optional doc_kind, adr_id, and top_k fields. Verify it by running eval.py against the live index.

Does changing queries.yaml require re-indexing the RAG database?

No, queries.yaml changes do not require re-indexing because they only define eval queries. Only metadata-rules.yaml changes require a force-full re-index, since doc_kind is embedded into chunks during ingest.

Why are my documents tagged with the wrong doc_kind?

Wrong doc_kind usually results from a broad glob placed before a specific one, since the first matching rule wins. Reorder rules so specific patterns come first, then run a force-full re-index to fix stale classifications.

When should I add a new adr_id pattern?

Add a new adr_id pattern only when the ADR file naming convention changes. The three built-in patterns already cover folder style, ADR-prefix style, and numeric-prefix style naming.