skills.rest
ZhangHanbo
ZhangHanboCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

benchmark-survey

Survey literature and extract structured benchmark metadata with baselines.

Updated Aug 23, 2026
Search Tags:#metrics#benchmark#installation#ranking#formalization#research-evaluation#literature-survey
One-click install
npx skills add https://github.com/ZhangHanbo/alpha_research --skill benchmark-survey
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: benchmark-survey
Source: https://github.com/ZhangHanbo/alpha_research/tree/main/skills/benchmark-survey
Command: npx skills add https://github.com/ZhangHanbo/alpha_research --skill benchmark-survey

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill automates the discovery and structured evaluation of benchmarks relevant to a project's formalized problem class, turning scattered literature and leaderboards into a prioritized set of candidate benchmarks and a written proposal for researcher review.

Core Features & Use Cases

  • Problem extraction: Summarizes the formalization's observation/action/task vocabulary to build search queries for relevant benchmarks.
  • Literature querying & extraction: Searches Papers With Code, alpha_review APIs, and surveys; reads benchmark papers and extracts task scope, metrics, baselines, install recipes, hardware needs, and community usage.
  • Ranking & recommendation: Scores candidates on coverage, non-saturation, community adoption, and install effort, then flags recommendations and produces a benchmark_proposal.md for human confirmation.
  • Use Case: Use during FORMALIZE to surface in-scope benchmarks before writing benchmarks.md or during APPROACH to check competitor evaluation coverage.

Quick Start

Run the benchmark-survey skill on your project directory with project.md and formalization.md to generate benchmark_proposal.md for human review.

Frequently Asked Questions about benchmark-survey

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find and rank research benchmarks for a formalized robotics problem?▼

To find research benchmarks, you can automate literature discovery to extract per-benchmark metadata and published baselines, then score candidates on community adoption and install effort to generate a ranked proposal.

What is the best way to extract standardized benchmark metrics from research papers?▼

Extracting benchmark metrics involves querying literature APIs and reading benchmark papers to pull structured metadata like task scope, canonical metrics, hardware requirements, and saturation trends for evaluation.

How do I check competitor evaluation coverage against existing benchmarks?▼

Checking competitor evaluation coverage requires applying formalized problem classes to identify in-scope and mismatched benchmarks, measuring community adoption, and comparing published baselines during the approach stage.

Can I automate benchmark discovery using formalization files?▼

Automating benchmark discovery uses your project and formalization files to summarize the observation and action vocabulary, building targeted search queries to surface relevant benchmarks before writing documentation.

What metrics are used to rank candidate benchmarks for a research project?▼

Ranking candidate benchmarks scores each option on coverage, non-saturation, community adoption, and installation effort, producing a weighted ranking that flags recommendations for human review.

When should I survey literature for candidate benchmarks during my research workflow?▼

Surveying literature for candidate benchmarks is recommended during the formalization stage to surface in-scope benchmarks early, or during the approach stage to verify competitor evaluation coverage.

Related Skills

Explore More Education & Research Skills→
tachyon-beeptachyon-beep
14

using-deep-rl

Route reinforcement learning tasks to specialized deep-RL skills by problem type.

Community
Advanced
gula00gula00
2

interview-designer

Design structured interview guides with role-specific scorecards and resume analysis.

Community
Advanced
daddiadaddia
1

write-metrics-report

Generate a metrics-report.md with delivery and quality metrics against baselines.

Community
Intermediate
EternalWaveeEternalWavee
36

benchmark-research-skill

Extract benchmarks, datasets, and metrics from research papers.

Community
Advanced