scholar-evaluation

Evaluate scholarly work across eight quality dimensions using the ScholarEval framework with structured scoring.

Updated Oct 7, 2022
One-click install
npx skills add https://github.com/tamagusko/linux-cfg --skill scholar-evaluation-tamagusko
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: scholar-evaluation
Source: https://github.com/tamagusko/linux-cfg/tree/main/dotfiles/claude/skills/scholar-evaluation
Command: npx skills add https://github.com/tamagusko/linux-cfg --skill scholar-evaluation-tamagusko

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Assessing research papers, proposals, and literature reviews consistently is difficult without a structured rubric, leading to vague or biased feedback. This Skill applies the ScholarEval framework to produce systematic, dimension-by-dimension evaluations with quantitative scores and actionable recommendations. ## Core Features & Use Cases - Eight-Dimension Assessment: Evaluates problem formulation, literature review, methodology, data collection, analysis, results, writing, and citations against detailed rubrics. - Quantitative Scoring: Calculates weighted aggregate scores on a 5-point scale with quality-level classification via the included Python script. - Actionable Feedback: Generates prioritized, evidence-based recommendations and publication-readiness assessments tailored to work stage, venue, and discipline. - Use Case: A researcher submits a draft machine learning paper and receives a structured report scoring each dimension, identifying that methodology is strong but the literature review lacks recent sources, with ranked revision priorities. ## Quick Start Evaluate this research paper using the ScholarEval framework and produce a scored report with prioritized feedback.

Frequently Asked Questions about scholar-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate a research paper with the ScholarEval framework?

Provide the paper and specify the evaluation scope (comprehensive, targeted, or comparative). The skill assesses eight dimensions including methodology, analysis, and writing, then synthesizes scores and prioritized recommendations into a structured report.

How do I calculate aggregate evaluation scores from dimension ratings?

Run scripts/calculate_scores.py with a JSON file of dimension scores using the --scores flag. It computes a weighted average, assigns a quality level, and generates a report with an ASCII bar chart; use --interactive for manual entry.

What dimensions does ScholarEval assess in academic work?

It covers problem formulation, literature review, methodology, data collection, analysis, results, scholarly writing, and citations. Each dimension has detailed rubrics in references/evaluation_framework.md with 5-point quality indicators.

Can I customize the dimension weights in the score calculation?

Yes, pass a custom weights JSON file with the --weights flag to calculate_scores.py. Weights must sum to 1.0; otherwise the script uses defaults where methodology weighs 20% and citations 5%.

When should I not use a standardized evaluation framework for peer review?

The framework complements but does not replace domain-specific expertise, and some dimensions do not apply to all work types, such as data collection for purely theoretical papers. Adjust expectations for work stage, venue, and disciplinary norms.