hugging-face-evaluation

Extract evaluation tables from Hugging Face READMEs and update model-index metadata.

1|Updated Jan 29, 2026

Search Tags:#benchmarking #model evaluation #model-index #leaderboards #model cards #hugging face

Ready to run

npx skills add https://github.com/yuxia214/-22 --skill hugging-face-evaluation-yuxia214

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/yuxia214/-22/tree/main/.cursor/skills/hugging-face-evaluation
Command: npx skills add https://github.com/yuxia214/-22 --skill hugging-face-evaluation-yuxia214

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface-hub, markdown-it-py, python-dotenv, pyyaml, requests, inspect-ai, inspect-evals, openai, lighteval[accelerate,vllm], vllm, torch, transformers, accelerate, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the process of adding and managing evaluation results for Hugging Face models, ensuring consistency and compatibility with the Hugging Face ecosystem.

Core Features & Use Cases

Automated Evaluation Updates: Extract evaluation tables from READMEs, import scores from external APIs, or run custom evaluations.
Model Card Integration: Automatically updates model-index metadata for leaderboards and compatibility.
Use Case: A researcher finishes evaluating their new language model on several benchmarks. They use this Skill to automatically parse the results from their README, format them into the model-index standard, and create a pull request to update the model card.

Quick Start

Use the hugging-face-evaluation skill to extract evaluation tables from the model's README file.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I update Hugging Face model cards with new benchmark evaluation results?▼

To update Hugging Face model cards with benchmark results, you can automatically extract evaluation tables from README files and format them into the model-index metadata standard. This ensures compatibility with leaderboards and Papers with Code.

Can I import evaluation scores from Artificial Analysis into a Hugging Face model card?▼

Yes, you can import evaluation scores directly from Artificial Analysis APIs into your Hugging Face model card. The tool parses these external scores and formats them into the required model-index metadata for leaderboards.

How do I run custom model evaluations using lighteval or vLLM for Hugging Face leaderboards?▼

You can run custom model evaluations using vLLM or lighteval by configuring the evaluation scripts. The results are automatically parsed and integrated into the model-index metadata within your Hugging Face model card.

Does this workflow require specific Python libraries for parsing markdown and interacting with the Hugging Face API?▼

Yes, the workflow requires huggingface-hub for API interactions and markdown-it-py for parsing README files. Additional dependencies like pyyaml, requests, and inspect-ai are also needed for full evaluation and metadata management.

What is the model-index metadata format used for Hugging Face leaderboards?▼

The model-index metadata format is a standardized YAML structure used in Hugging Face model cards to record benchmark evaluation results. It enables automatic integration with Hugging Face leaderboards and Papers with Code compatibility.

Why are my Hugging Face model evaluation tables not showing up on leaderboards?▼

Evaluation tables may not appear on leaderboards if they are not formatted into the correct model-index metadata standard. This tool extracts tables from your README and updates the model card metadata to ensure proper leaderboard integration.