Run retrieval and injection evaluation tests on synthetic and real data.
npx skills add https://github.com/emmahyde/memesis --skill run-eval-emmahyde
Please help me install this Agent Skill. Skill: run-eval Source: https://github.com/emmahyde/memesis/tree/main/skills/run-eval Command: npx skills add https://github.com/emmahyde/memesis --skill run-eval-emmahyde
This Skill simplifies the process of running evaluation suites to measure the accuracy and performance of retrieval and injection mechanisms within memory-driven AI systems.
Ask the AI to run the evaluation suite to check current retrieval performance on your dataset.
High-intent search queries and answers about installing and using this skill.
To run an evaluation suite for AI model retrieval accuracy, you prompt the AI to execute the evaluation pipeline on your dataset. It automates testing of retrieval and injection mechanisms to measure performance metrics.
Regression comparison in AI memory systems is the process of capturing baseline performance metrics and comparing them against current test results. It helps detect if recent changes have degraded model retrieval accuracy or injection quality.
Yes, you can use synthetic data to assess retrieval quality in memory-driven AI. The evaluation tests support both synthetic and real observation benchmarks to comprehensively measure model robustness and accuracy.
To check if recent changes reduced retrieval accuracy, run the full evaluation pipeline with minimal command input. It compares current test results against captured baseline metrics to immediately identify any performance regressions.
No specific dependencies are required to perform model evaluation tests. The Skill operates independently to handle baseline measurement, regression comparison, and full pipeline execution for quality assurance in AI development.
The best way to measure injection quality in AI memory systems is by running comprehensive evaluation pipelines. This approach automates synthetic and real observation benchmarks to ensure model robustness and verify retrieval accuracy.
shekerkammaMeasure Core Web Vitals and resource sizes to detect gradual performance regressions.
as4584Compare embedding models and implement pipelines with chunking and retrieval metrics.
rohunvoraAutomate LLM-as-a-judge scoring, pairwise comparison, and rubric generation with bias mitigation.
ISAQQSAIAutomate LLM evaluation with metrics, human feedback, and benchmarking templates.
