model-eval-report

What problem does it solve?

Generates a professional Quarto HTML model performance evaluation report for the CRPD Dashboard LLM integration, enabling consistent, shareable assessments from the latest evaluation results.

Core Features & Use Cases

• Automatically locate the latest evaluation results JSON from LLM_Development and extract phase, run_date, test_questions, configs, winner, and recommendation to drive report generation.
• Generate a polished Quarto (.qmd) report with interactive visuals, a performance summary, and a model recommendation.
• Save reports sequentially in Model Performance Evaluation/ as report_001_, report_002_, ... with a phase slug and date in the filename.
• Trigger on demand or automatically after phase evaluation completes (evaluate_phase*.py) when a report for that phase hasn't been generated yet.
• Handle multiple configs and rate-limit skipped configurations, including a clear recommendation and known gaps.

Quick Start

Ask the skill to generate the latest model evaluation report and it will locate the most recent eval_results_phase*.json, build a Quarto report, and save it under Model Performance Evaluation/report_NNN_phaseSlug_YYYY-MM-DD.qmd.