Automate end-to-end evaluation of Zep memory retrieval and QA capabilities.
npx skills add https://github.com/getzep/zep --skill zep-eval-harness
Please help me install this Agent Skill. Skill: zep-eval-harness Source: https://github.com/getzep/zep/tree/main/zep-eval-harness/.claude/commands/zep-eval-harness Command: npx skills add https://github.com/getzep/zep --skill zep-eval-harness
Zep Eval Harness provides an end-to-end evaluation framework to test Zep's memory retrieval and QA capabilities by feeding conversations, telemetry, and documents into graphs and scoring retrieved contexts against gold answers.
Run the Zep eval harness end-to-end to chunk documents, ingest users and documents, run evaluations, inspect graphs, and analyze results.
High-intent search queries and answers about installing and using this skill.
You can evaluate graph-based memory retrieval by feeding conversations, telemetry, and documents into graphs, then scoring retrieved contexts against gold answers to validate retrieval quality and response correctness.
Memory retrieval scoring focuses on Context Complete as the primary metric to assess retrieval quality, and Answer Accuracy as the secondary metric to evaluate response correctness against golden solutions.
You can inspect graphs by using dedicated scripts to view graph contents, allowing you to diagnose retrieval failures and identify missing context nodes within your ingested conversations and documents.
Document chunking splits source documents into manageable chunks and automatically generates contextualized summaries, preparing the text data for ingestion into standalone graphs for LLM evaluation.
You can compare evaluation runs by analyzing aggregate and per-category metrics stored in the runs directories, enabling side-by-side analysis of retrieval performance across different configurations.
angryracoonPrepare Salesforce Data Cloud ingestion assets for raw source data.
opendatahub-ioAnalyze evaluation results to identify alignment gaps between automated checks and human assessment.
pax-kExport Obsidian wiki wikilinks into graph.json, graph.graphml, cypher.txt, and graph.html.
mindmorassBenchmark embedding models on document, code, and QA datasets.
