report

Generate LongMemEval benchmark reports in markdown, JSON, or summary formats.

3|1|Updated Apr 11, 2026

Search Tags:#benchmarking #report generation #scorecard #accuracy analysis #LongMemEval

Ready to run

npx skills add https://github.com/tmuskal/arc-agi-benchmarker --skill report-tmuskal

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: report
Source: https://github.com/tmuskal/arc-agi-benchmarker/tree/main/plugins/longmemeval-benchmarker/skills/report
Command: npx skills add https://github.com/tmuskal/arc-agi-benchmarker --skill report-tmuskal

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill automatically generates a detailed performance report from completed LongMemEval runs, including overall and per-question-type accuracy.

Core Features & Use Cases

Generate Report: Automatically compute and produce a comprehensive scorecard from LongMemEval runs.
Customize Format: Output the report in various formats: markdown, JSON, or a summary.
Run Identification: Can be applied to a specific run ID or the latest one available.
Use Case: Imagine you have conducted several LongMemEval tests and need a detailed breakdown of the results for analysis or review. This Skill would compile all relevant information into a concise and actionable report.

Quick Start

Run the command /arc-agi-benchmarker:report latest to generate a scorecard from the latest LongMemEval run in markdown format.

Frequently Asked Questions about report

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I generate a scorecard from LongMemEval benchmark results?▼

Generate a LongMemEval scorecard by running the report command against your JSON-structured run data to automatically compute overall and per-question-type accuracy into a detailed scorecard.

Can I export LongMemEval accuracy analysis results in JSON format?▼

Yes, you can export LongMemEval accuracy analysis results in JSON format by customizing the output format setting to include markdown, JSON, or a summary during the report generation process.

What is the best way to analyze accuracy from a specific LongMemEval run?▼

To analyze accuracy from a specific LongMemEval run, provide the target run ID to the report generator to render detailed benchmark results for that particular test instead of the latest one.

Does the LongMemEval report generator require any external dependencies?▼

No, the LongMemEval report generator operates without external dependencies, using internal scripts to parse JSON-structured data and render benchmark scorecards directly within your environment.

What data format is needed to produce a LongMemEval performance report?▼

Producing a LongMemEval performance report requires JSON-structured data from completed benchmarking runs to accurately compute and render the overall and per-question-type accuracy metrics.