phd-benchmark-paper-template

Structures benchmark and evaluation papers using a five-pillar framework with section skeletons and checklists.

52|Updated Apr 24, 2026
One-click install
npx skills add https://github.com/Immortalqx/my_codex_skills --skill phd-benchmark-paper-template-immortalqx
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: phd-benchmark-paper-template
Source: https://github.com/Immortalqx/my_codex_skills/tree/main/phd-benchmark-paper-template
Command: npx skills add https://github.com/Immortalqx/my_codex_skills --skill phd-benchmark-paper-template-immortalqx

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Writing a benchmark or evaluation paper requires a fundamentally different structure than a technical paper, and researchers often miss critical elements reviewers expect, such as a construction pipeline figure, a benchmark comparison table, or bolded Finding summaries. This Skill audits a benchmark idea against a five-pillar framework and produces the complete logic chain and section skeleton before writing begins. ## Core Features & Use Cases - Five-Pillar Completeness Audit: Checks whether the Research Gap, Construction Pipeline, Evaluation Framework, Empirical Findings, and optional Companion Method are each articulated, with improvement suggestions for gaps. - Introduction Six-Part Logic Chain: Generates the Background plus Running Example, Existing-Benchmark Limitations, Research Questions, Design Considerations, Proposal, and Contributions structure specific to benchmark papers. - Section Skeleton and Pre-Submission Checklist: Produces a Section 2-7 outline with figure and table placement, plus a four-category reviewer checklist with Critical, Major, and Minor severity classification. - Use Case: A researcher planning a Text-to-SQL benchmark submits their idea, receives a completeness table showing their evaluation framework lacks a fine-grained taxonomy, and gets a full section skeleton with the pipeline figure and comparison table planned before data construction starts. ## Quick Start Use the phd-benchmark-paper-template skill to audit my benchmark paper idea on ambiguity in code generation and produce the five-pillar completeness table, Introduction logic chain, and section skeleton.

Frequently Asked Questions about phd-benchmark-paper-template

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I structure a benchmark paper for NeurIPS or ICLR?

Organize the paper around five pillars: research gap, construction pipeline, evaluation framework, empirical findings, and an optional companion method. The Introduction follows a six-part chain from running example through contributions, and Sections 2-7 cover task design, pipeline, experiments by RQ, discussion, related work, and conclusion.

What is the difference between a benchmark paper and a technical paper?

A technical paper's main contribution is a novel algorithm, while a benchmark paper's contribution is a new evaluation dimension or dataset. The benchmark paper's heaviest chapter is the construction pipeline rather than the method, and its experiments reveal model capability boundaries instead of proving one method beats baselines.

How do I check if my benchmark idea is substantive enough?

Validate the gap against four checks: practical impact, measurability, non-triviality, and actionability. The gap must be a structural limitation of existing evaluation, such as dimension blindness, assumption violation, or granularity mismatch, not simply a need for more data.

What figures and tables does a benchmark paper require?

Four items are mandatory: Figure 1 as a running example illustrating the evaluation gap, Table 1 comparing against at least five existing benchmarks, Figure 2 showing the construction pipeline with data counts and quality gates, and a large overall performance table covering all models and metrics.

When should I not use this benchmark paper template?

Do not use it for technical or position papers, since the Introduction flowchart and section structure differ fundamentally. For technical papers use a technical-paper template, and for drafting only an Introduction in isolation use a dedicated introduction-drafting skill.