recall-evals

Evaluate knowledge base recall precision by scoring precision@k, hit-rate, and reciprocal rank metrics against golden set facts. This Skill computes these measurements using existing recall paths to ensure high recall of durable, non-PHI facts.

Yes, you can measure recall latency without affecting the memory ledger. This evaluation harness operates strictly on non-PHI facts and existing recall paths, ensuring the memory ledger remains completely unaffected during testing.

Memory performance evaluation uses precision@k, hit-rate, and Mean Reciprocal Rank (MRR) metrics. These scoring functions compute measurements for each query based on known durable facts and existing recall paths.

Yes, the recall evaluation process hard-guards against PHI content by operating exclusively in the Non-PHI ring. It evaluates only non-PHI facts, ensuring protected health information is completely excluded from all measurements.

The best way to run a recall quality measurement is executing the 'node skills/recall-evals/run-evals.mjs' script via the 'skills/kb-memory/run.sh' entry point to score precision, hit-rate, and MRR for your knowledge base.

Yes, you can get measurement-based suggestions to improve knowledge base recall. The harness measures memory and recall changes over time, suggesting data-driven improvements based on the computed precision and hit-rate metrics.