hugging-face-evaluation

Name: hugging-face-evaluation
Availability: InStock
Author: Yog-Sotho

Extract eval tables from Hugging Face READMEs into model-index YAML.

4|Updated Mar 21, 2026

Search Tags:#uv #evaluation #benchmark #huggingface #model-index #hf-jobs #model-card

Ready to run

npx skills add https://github.com/Yog-Sotho/claude-skills --skill hugging-face-evaluation-yog-sotho

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/Yog-Sotho/claude-skills/tree/main/hugging-face-evaluation
Command: npx skills add https://github.com/Yog-Sotho/claude-skills --skill hugging-face-evaluation-yog-sotho

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Add, import, and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing benchmark scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval/inspect-ai on HF Jobs or locally. Works with the model-index metadata format for leaderboard and Papers with Code integration.

Core Features & Use Cases

Extract eval tables from README content and convert them to model-index YAML.
Import benchmark scores from Artificial Analysis API.
Run model evaluations on HF Jobs or locally using vLLM, lighteval, and inspect-ai.
Integrates with model-index metadata format for leaderboard and Papers with Code.

Quick Start

Run an evaluation workflow on a Hugging Face model card using uv and the evaluation_manager tools.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I extract evaluation tables from a Hugging Face model card README?▼

You can extract evaluation tables from Hugging Face model card READMEs by using script-driven extraction tools that parse the content and convert the benchmark results into model-index YAML format.

Can I import benchmark scores from Artificial Analysis into a Hugging Face model card?▼

Yes, you can import benchmark scores from Artificial Analysis into Hugging Face model cards by using the Artificial Analysis API integration, which requires the AA_API_KEY environment variable to fetch and sync the data.

How do I run model evaluations with vLLM or lighteval on HF Jobs?▼

You can run model evaluations with vLLM or lighteval on HF Jobs by using the evaluation_manager tools via uv, executing custom eval tasks either directly on HF Jobs or through local runtimes using inspect-ai.

Does this workflow support the model-index metadata format for Papers with Code integration?▼

Yes, this workflow fully supports the model-index metadata format, allowing you to generate structured YAML outputs that integrate seamlessly with leaderboards and Papers with Code.

What environment variables do I need to set up to manage Hugging Face model card evaluations?▼

To manage Hugging Face model card evaluations, you need to set environment variables such as HF_TOKEN for Hugging Face API access and AA_API_KEY for Artificial Analysis benchmark imports.

Why convert extracted evaluation tables to model-index YAML for Hugging Face leaderboards?▼

Converting extracted evaluation tables to model-index YAML standardizes the benchmark results, ensuring compatibility with Hugging Face leaderboards and enabling automated Papers with Code integration.