skills.rest
surus-lat
surus-latOFFICIAL
Explore all skills by this author →
Website𝕏@surusdevGitHub

rebuild-leaderboard

Automate benchmark re-evaluation and publish results to HuggingFace datasets.

8|Updated Sep 12, 2025
Search Tags:#automation#benchmarking#reproducibility#cluster-management#huggingface#ai-evaluation
One-click install
npx skills add https://github.com/surus-lat/benchy --skill rebuild-leaderboard
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: rebuild-leaderboard
Source: https://github.com/surus-lat/benchy/tree/main/.agent/skills/rebuild-leaderboard
Command: npx skills add https://github.com/surus-lat/benchy --skill rebuild-leaderboard

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill addresses the critical need to restore and re-verify benchmark results on the LatamBoard after data loss or cluster failure, ensuring consistent model performance tracking.

Core Features & Use Cases

  • Automated Re-evaluation: Orchestrates the full benchmark suite for multiple models across GPU clusters.
  • Incremental Publishing: Merges and publishes results to HuggingFace after each model run to prevent data loss.
  • Use Case: If a cluster wipe occurs, use this skill to systematically re-run all models, validate their configurations, and restore the leaderboard state without manual intervention for every step.

Quick Start

Use the rebuild-leaderboard skill to initiate a full re-evaluation of all models currently listed in the configuration directory.

Frequently Asked Questions about rebuild-leaderboard

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I restore a HuggingFace leaderboard after a GPU cluster failure?▼

Restore a HuggingFace leaderboard by orchestrating an automated re-evaluation of all models on the GPU cluster. This skill manages task execution and synchronizes results incrementally to HuggingFace datasets to prevent data loss during recovery.

What is the best way to automate AI benchmarking across multiple models?▼

Automate AI benchmarking by orchestrating the full benchmark suite across available GPU clusters. This skill handles execution and resource allocation while merging results to HuggingFace after each model run to ensure reproducible performance tracking.

Do I need a specific CLI tool to run automated benchmark recovery?▼

Yes, automated benchmark recovery requires the benchy CLI to manage task execution. You also need a configured GPU cluster and valid HuggingFace authentication tokens to successfully synchronize the restored evaluation results.

Can I prevent data loss when re-running large benchmark suites on a GPU cluster?▼

Prevent data loss when re-running benchmarks by using incremental publishing. This skill merges and publishes results to HuggingFace datasets immediately after each individual model run, ensuring progress is saved if a subsequent failure occurs.

Does this automated benchmarking workflow require manual intervention for each model?▼

No, the automated benchmarking workflow reduces manual intervention by systematically reading all model configurations from the directory and executing the full suite. It handles end-to-end execution and publication automatically for every listed model.

Related Skills

Explore More Software Engineering Skills→
davpatel605-beepdavpatel605-beep

stripe-projects

Provision SaaS infrastructure and synchronize credentials via Stripe CLI Projects.

Community
Intermediate
surus-latsurus-lat
8

push-to-latamboard

Merge local benchmark results into the HuggingFace LatamBoard leaderboard dataset.

Official
Advanced
Supporter09Supporter09

evaluating-cosmos-policy

Runs Cosmos Policy LIBERO and RoboCasa benchmark evaluations via command-line workflows.

Community
Advanced
u9401066u9401066
2

mcp-ml-training

Train ML models end-to-end with MCP AutoML, including data upload, model selection, and evaluation.

Community
Advanced