skills.rest
yuxia214
yuxia214COMMUNITY
Explore all skills by this author →
WebsiteGitHub

hugging-face-evaluation

Extract evaluation tables from Hugging Face READMEs and update model-index metadata.

1|Updated Jan 29, 2026
Search Tags:#benchmarking#model evaluation#model-index#leaderboards#model cards#hugging face
One-click install
npx skills add https://github.com/yuxia214/-22 --skill hugging-face-evaluation-yuxia214
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/yuxia214/-22/tree/main/.cursor/skills/hugging-face-evaluation
Command: npx skills add https://github.com/yuxia214/-22 --skill hugging-face-evaluation-yuxia214

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface-hub, markdown-it-py, python-dotenv, pyyaml, requests, inspect-ai, inspect-evals, openai, lighteval[accelerate,vllm], vllm, torch, transformers, accelerate, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the process of adding and managing evaluation results for Hugging Face models, ensuring consistency and compatibility with the Hugging Face ecosystem.

Core Features & Use Cases

  • Automated Evaluation Updates: Extract evaluation tables from READMEs, import scores from external APIs, or run custom evaluations.
  • Model Card Integration: Automatically updates model-index metadata for leaderboards and compatibility.
  • Use Case: A researcher finishes evaluating their new language model on several benchmarks. They use this Skill to automatically parse the results from their README, format them into the model-index standard, and create a pull request to update the model card.

Quick Start

Use the hugging-face-evaluation skill to extract evaluation tables from the model's README file.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I update Hugging Face model cards with new benchmark evaluation results?▼

To update Hugging Face model cards with benchmark results, you can automatically extract evaluation tables from README files and format them into the model-index metadata standard. This ensures compatibility with leaderboards and Papers with Code.

Can I import evaluation scores from Artificial Analysis into a Hugging Face model card?▼

Yes, you can import evaluation scores directly from Artificial Analysis APIs into your Hugging Face model card. The tool parses these external scores and formats them into the required model-index metadata for leaderboards.

How do I run custom model evaluations using lighteval or vLLM for Hugging Face leaderboards?▼

You can run custom model evaluations using vLLM or lighteval by configuring the evaluation scripts. The results are automatically parsed and integrated into the model-index metadata within your Hugging Face model card.

Does this workflow require specific Python libraries for parsing markdown and interacting with the Hugging Face API?▼

Yes, the workflow requires huggingface-hub for API interactions and markdown-it-py for parsing README files. Additional dependencies like pyyaml, requests, and inspect-ai are also needed for full evaluation and metadata management.

What is the model-index metadata format used for Hugging Face leaderboards?▼

The model-index metadata format is a standardized YAML structure used in Hugging Face model cards to record benchmark evaluation results. It enables automatic integration with Hugging Face leaderboards and Papers with Code compatibility.

Why are my Hugging Face model evaluation tables not showing up on leaderboards?▼

Evaluation tables may not appear on leaderboards if they are not formatted into the correct model-index metadata standard. This tool extracts tables from your README and updates the model card metadata to ensure proper leaderboard integration.

Related Skills

Explore More Software Engineering Skills→
aisa-groupaisa-group
90

hugging-face-evaluation-manager

Manage structured evaluation results in Hugging Face model cards.

Official
Advanced
3kpro3kpro

hugging-face-evaluation

Manage evaluation results in Hugging Face model cards via READMEs, Artificial Analysis, vLLM, or lighteval.

Community
Advanced
FacuMFacuM

hugging-face-evaluation

Extract evaluation tables from Hugging Face model README files.

Community
Advanced
keremtoker468-dotcomkeremtoker468-dotcom

hugging-face-evaluation

Extract evaluation tables from Hugging Face model README files.

Community
Advanced