recall-evals

Name: recall-evals
Availability: InStock
Author: InnerScopeHearing

Evaluate recall precision and latency using golden set facts.

Updated May 30, 2026

Search Tags:#knowledge base #recall evaluation #precision measurement #memory performance #PHI exclusion

Ready to run

npx skills add https://github.com/InnerScopeHearing/otchealth-claude-tools --skill recall-evals

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: recall-evals
Source: https://github.com/InnerScopeHearing/otchealth-claude-tools/tree/main/skills/recall-evals
Command: npx skills add https://github.com/InnerScopeHearing/otchealth-claude-tools --skill recall-evals

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a recall-quality eval harness for the knowledge base, ensuring high precision and recall of durable, non-PHI facts without affecting the ledger or memory.

Core Features & Use Cases

Evaluation of Recall Quality: Scores precision@k, hit-rate, and MRR for known durable facts.
Non-PHI ring; PHI-excluded: Operates only in the Non-PHI ring, hard-guarding against PHI content.
Measurement-based Adjustments: Measures memory/recall changes and suggests improvements based on data.

Quick Start

Use the 'skills/kb-memory/run.sh' script to run the recall-evals skill for measuring recall quality with 'node skills/recall-evals/run-evals.mjs'.

Frequently Asked Questions about recall-evals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I evaluate knowledge base recall precision for durable facts?▼

Evaluate knowledge base recall precision by scoring precision@k, hit-rate, and reciprocal rank metrics against golden set facts. This Skill computes these measurements using existing recall paths to ensure high recall of durable, non-PHI facts.

Can I measure recall latency without affecting the memory ledger?▼

Yes, you can measure recall latency without affecting the memory ledger. This evaluation harness operates strictly on non-PHI facts and existing recall paths, ensuring the memory ledger remains completely unaffected during testing.

What metrics are used for memory performance evaluation?▼

Memory performance evaluation uses precision@k, hit-rate, and Mean Reciprocal Rank (MRR) metrics. These scoring functions compute measurements for each query based on known durable facts and existing recall paths.

Does the recall evaluation process exclude PHI content?▼

Yes, the recall evaluation process hard-guards against PHI content by operating exclusively in the Non-PHI ring. It evaluates only non-PHI facts, ensuring protected health information is completely excluded from all measurements.

What's the best way to run a recall quality measurement?▼

The best way to run a recall quality measurement is executing the 'node skills/recall-evals/run-evals.mjs' script via the 'skills/kb-memory/run.sh' entry point to score precision, hit-rate, and MRR for your knowledge base.

Can I get measurement-based suggestions to improve knowledge base recall?▼

Yes, you can get measurement-based suggestions to improve knowledge base recall. The harness measures memory and recall changes over time, suggesting data-driven improvements based on the computed precision and hit-rate metrics.