skills.rest
FacuM
FacuMCOMMUNITY
Explore all skills by this author →
GitHub

hugging-face-evaluation

Extract evaluation tables from Hugging Face model README files.

1|Updated Feb 24, 2026
Search Tags:#benchmarking#evaluation#inspect-ai#vllm#leaderboards#model cards#hugging face
One-click install
npx skills add https://github.com/FacuM/yolo-agent --skill hugging-face-evaluation-facum
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/FacuM/yolo-agent/tree/main/.claude/skills/hugging-face-evaluation
Command: npx skills add https://github.com/FacuM/yolo-agent --skill hugging-face-evaluation-facum

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface-hub, markdown-it-py, python-dotenv, pyyaml, requests, inspect-ai, inspect-evals, openai, lighteval[accelerate,vllm], vllm, torch, transformers, accelerate, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill streamlines the process of adding and managing evaluation results for Hugging Face models, ensuring accurate and up-to-date benchmark data is reflected in model cards.

Core Features & Use Cases

  • Extract from README: Parses evaluation tables directly from model README files.
  • Import from Artificial Analysis: Fetches benchmark scores via API.
  • Run Evaluations: Integrates with Hugging Face Jobs and vLLM for custom evaluations.
  • Use Case: Automatically update your model's benchmark scores on Hugging Face Hub after running new evaluations, or contribute benchmark results to community models by extracting tables from their READMEs.

Quick Start

Use the hugging-face-evaluation skill to extract evaluation tables from the model's README file.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract evaluation benchmark tables from a Hugging Face model's README?▼

To extract evaluation benchmark tables from a Hugging Face README, use this skill to parse model-index metadata directly from the README files. It automatically fetches and structures the benchmark scores for leaderboard compatibility.

Can I import benchmark scores from the Artificial Analysis API into Hugging Face model cards?▼

Yes, you can import benchmark scores from the Artificial Analysis API into Hugging Face model cards. The skill fetches evaluation results via the API and formats them into the model-index metadata structure for your model cards.

How do I run custom LLM evaluations using vLLM and Hugging Face Jobs?▼

You can run custom LLM evaluations using vLLM and Hugging Face Jobs by leveraging this skill's integration with inspect-ai and lighteval frameworks. It manages the execution and updates the evaluation results automatically.

Does this Hugging Face evaluation tool support the inspect-ai framework?▼

Yes, this Hugging Face evaluation tool supports the inspect-ai framework. It integrates with inspect-ai and lighteval to run custom evaluations and manage benchmark scores within the model-index metadata format.

What is the best way to update Hugging Face model cards with new benchmark results?▼

The best way to update Hugging Face model cards with new benchmark results is by formatting scores into the model-index metadata. This skill extracts data from READMEs or APIs and ensures accurate, leaderboard-compatible updates.

Do I need vLLM and lighteval installed to manage Hugging Face model evaluations?▼

You need vLLM and lighteval installed if you intend to run custom evaluations directly. However, you can extract existing tables from READMEs or fetch scores via the Artificial Analysis API without requiring these inference dependencies.

Related Skills

Explore More Software Engineering Skills→
BlackRoad-OS-IncBlackRoad-OS-Inc
1

hugging-face-evaluation

Consolidate README benchmarks and Artificial Analysis imports into Hugging Face model-index entries.

Official
Advanced
yuxia214yuxia214

hugging-face-evaluation

Extract evaluation tables from Hugging Face READMEs and update model-index metadata.

Community
Advanced
niznik-devniznik-dev
16

create-inspect-task

Create custom inspect-ai evaluation tasks with experiment-guided and standalone modes.

Community
Intermediate
POWERFULMOVESPOWERFULMOVES
1

hugging-face-evaluation-manager

Extract evaluation tables from READMEs and import benchmark scores from the Artificial Analysis API.

Community
Advanced