Classify batch evaluation results and generate publication-ready markdown summary tables.
npx skills add https://github.com/SamyakJhaveri/loam --skill eval-grader
Please help me install this Agent Skill. Skill: eval-grader Source: https://github.com/SamyakJhaveri/loam/tree/main/seed/_research/skills/eval-grader Command: npx skills add https://github.com/SamyakJhaveri/loam --skill eval-grader
Automates the grading and classification of evaluation results from batch runs, producing defensible statistics for paper reporting.
Run the evaluator on a results directory to classify outcomes and generate a summary table
High-intent search queries and answers about installing and using this skill.
Classifying model failure modes uses a structured taxonomy to categorize errors from evaluation runs. It diagnoses failure patterns by applying exclusions for known issues, ensuring defensible and accurate per-model statistics for reporting.
Excluding known issues from model evaluation statistics requires applying a known-issues list to loaded batch run results. This workflow filters out predefined failures before computing pass/fail statistics, ensuring reported outcomes reflect genuine model performance.
Publication-ready summary tables require structured failure classification and computed pass/fail statistics from batch runs. By loading results, applying exclusions, and categorizing failures, the workflow produces defensible markdown tables suitable for papers.
Diagnosing failure patterns is supported through structured failure mode classification. The workflow categorizes errors from batch runs using a defined taxonomy, helping identify and quantify specific failure types for model evaluation reporting.
Quantifying pass/fail statistics for model evaluation experiments is best handled by automating the grading workflow. It loads results, applies exclusions for known issues, classifies outcomes, and computes defensible statistics for paper reporting.
ovachieverAutomate QA planning, test-suite generation, and coverage analysis for React, Next.js, and Node.js applications.
gabfssilvaGenerate PRISMA flowcharts and synthesis reports for systematic literature reviews.
HeXiao-55Monitor remote experiment runs via SSH and summarize JSON results.
breethomasAnalyze LLM pipeline traces to categorize failures and prioritize root causes.
