Run automated chunk-size sweeps and report nDCG, MRR, and Hit@5 metrics.
npx skills add https://github.com/nicholasglazer/gnosis-mcp --skill tune-nicholasglazer
Please help me install this Agent Skill. Skill: tune Source: https://github.com/nicholasglazer/gnosis-mcp/tree/main/skills/tune Command: npx skills add https://github.com/nicholasglazer/gnosis-mcp --skill tune-nicholasglazer
Chunking and retrieval configuration determine search quality; this skill helps you measure and optimize chunk size, embedding, and reranker settings against your corpus.
Run a default sweep to compare chunk sizes and retrieve settings for your corpus.
High-intent search queries and answers about installing and using this skill.
To tune chunking and retrieval settings, you can run automated sweeps across your corpus to compare different chunk sizes, ingestion states, and modes. The skill evaluates configurations using a golden-query set and reports nDCG, MRR, and Hit@5 metrics to identify optimal parameters.
A golden-query set is a benchmark dataset used to evaluate retrieval accuracy. You need it to perform ingest-based evaluation, as the skill uses this set to measure and report metrics like nDCG, MRR, and Hit@5, guiding your chunking configuration decisions.
You can find the best chunk size for your corpus by running an automated chunk-size sweep. This process tests various configurations across your corpus to identify the optimal balance of context and latency, measuring search quality with nDCG, MRR, and Hit@5 metrics.
Yes, you can evaluate embedding and reranker parameters against your own corpus. The skill applies automated sweeps across different ingestion states and modes, using benchmarks to maximize search quality and guide your configuration decisions.
The metrics used for evaluating retrieval quality during chunking sweeps are nDCG, MRR, and Hit@5. These metrics are reported after running automated sweeps against a golden-query set to help you measure and optimize corpus search performance.
The main limitation is that automated sweeps require a prepared golden-query set and enforce specific prerequisites before execution. You must have your corpus ingested and a benchmark dataset ready to measure nDCG, MRR, and Hit@5 metrics effectively.
rocklambrosEvaluate RAG pipelines with retrieval metrics, generation scores, and failure attribution.
Prompt-or-Die-LabsBuild elizaOS knowledge bases from domain documents with RAG configuration.
melikhanmutluBuild retrieval-augmented generation pipelines with vector databases and semantic search.
nicholasglazerEvaluate corpus retrieval quality with Hit@5, MRR, and nDCG metrics.
