What problem does it solve? Retrieval quality in an MCP server degrades silently when catalog descriptions, response shapes, search rankings, and golden evaluation answers drift from live service behavior. This Skill provides a disciplined, evidence-gated audit workflow so changes to the stellar-raven-codemode retrieval system are measured against real endpoint calls instead of intuition. ## Core Features & Use Cases - Surface Inventory: Builds a ledger of every exposed operation and skill, comparing the committed manifest, generated specification, adapters, live catalog, and fresh-client MCP descriptions. - Live Probing: Probes read-only operations with representative inputs, classifying responses as data, soft-empty, or error, and recording schema differences without storing secrets or raw bodies. - Experiment Gating: Requires pre-registered hypotheses, symmetric controls, cost caps, and independent adversarial review before any paid evaluation run or product change ships. - Use Case: An engineer suspects search ranking regressed after a catalog description edit. The Skill walks them through pinning a baseline, tracing retrieval decisions, running A/B evals with identical case identifiers, and shipping only changes that survive two unrelated targets. ## Quick Start Use the retrieval-system-audit skill to inventory every exposed operation, probe live response contracts, and pre-register an experiment for the suspected search ranking regression.
