skills.rest
LinusConradM
LinusConradMCOMMUNITY
Explore all skills by this author →
GitHub

model-eval-report

Generate a Quarto report from the latest LLM evaluation results JSON.

Updated Mar 27, 2026
Search Tags:#llm#evaluation#json#visualization#report#quarto#model-evaluation
One-click install
npx skills add https://github.com/LinusConradM/test_crpd_dashboard --skill model-eval-report
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-eval-report
Source: https://github.com/LinusConradM/test_crpd_dashboard/tree/main/.claude/skills/model-eval-report
Command: npx skills add https://github.com/LinusConradM/test_crpd_dashboard --skill model-eval-report

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Generates a professional Quarto HTML model performance evaluation report for the CRPD Dashboard LLM integration, enabling consistent, shareable assessments from the latest evaluation results.

Core Features & Use Cases

  • Automatically locate the latest evaluation results JSON from LLM_Development and extract phase, run_date, test_questions, configs, winner, and recommendation to drive report generation.
  • Generate a polished Quarto (.qmd) report with interactive visuals, a performance summary, and a model recommendation.
  • Save reports sequentially in Model Performance Evaluation/ as report_001_, report_002_, ... with a phase slug and date in the filename.
  • Trigger on demand or automatically after phase evaluation completes (evaluate_phase*.py) when a report for that phase hasn't been generated yet.
  • Handle multiple configs and rate-limit skipped configurations, including a clear recommendation and known gaps.

Quick Start

Ask the skill to generate the latest model evaluation report and it will locate the most recent eval_results_phase*.json, build a Quarto report, and save it under Model Performance Evaluation/report_NNN_phaseSlug_YYYY-MM-DD.qmd.

Frequently Asked Questions about model-eval-report

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I auto-generate an LLM model evaluation report from JSON results?▼

To generate an LLM model evaluation report from JSON, the skill locates the latest eval_results JSON in LLM_Development, extracts test metrics and configurations, and outputs a structured Quarto document with an executive summary and recommendation.

What metrics are included in a Quarto model evaluation report?▼

A Quarto model evaluation report includes performance metrics such as MRR and ndCG, alongside retrieval and judge-lens analyses, a performance summary table, model recommendations, and known gaps extracted from the evaluation JSON.

How do I create a Quarto report from a JSON file for LLM development?▼

You can create a Quarto report from a JSON file by having the skill identify the latest phase evaluation results, compute the next sequential report number, and save a formatted .qmd file with visualizations to the Model Performance Evaluation directory.

Can I generate a model performance evaluation report automatically after a phase evaluation completes?▼

Yes, the report generation can be triggered on demand or automatically after phase evaluation scripts finish, provided a report for that specific phase has not yet been generated.

How does the reporting tool handle rate-limit skipped configurations in LLM evaluation?▼

The reporting tool handles rate-limit skipped configurations by documenting them within the generated Quarto report, providing a clear model recommendation while explicitly noting the skipped configs as known gaps.

Do I need Quarto installed to generate model evaluation reports from JSON?▼

Quarto is the target format for the report output, generating a .qmd file with interactive visuals and structured analyses, so you need a Quarto environment to render the final HTML document.

Related Skills

Explore More Data & Analytics Skills→
maxvaegamaxvaega
146

json-parser

Parse and validate JSON data against schemas and JSONPath queries.

Community
Intermediate
wolf5996wolf5996
5

md-to-html

Convert Markdown drafts into self-contained HTML via Quarto rendering.

Community
Intermediate
clinecline
19

skill-creator

Create and iteratively improve AI agent skills through structured evaluation workflows.

Official
Advanced
yennanliuyennanliu
164

Report Generator

Generate HTML and PDF investment reports with interactive visualizations and data tables.

Community
Advanced