skills.rest
keremtoker468-dotcom
keremtoker468-dotcomCOMMUNITY
Explore all skills by this author →
GitHub

hugging-face-evaluation

Extract evaluation tables from Hugging Face model README files.

Updated Feb 26, 2026
Search Tags:#benchmarking#evaluation#huggingface#vllm#leaderboard#lighteval#model card
One-click install
npx skills add https://github.com/keremtoker468-dotcom/restoran --skill hugging-face-evaluation-keremtoker468-dotcom
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/keremtoker468-dotcom/restoran/tree/main/.claude/skills/hugging-face-evaluation
Command: npx skills add https://github.com/keremtoker468-dotcom/restoran --skill hugging-face-evaluation-keremtoker468-dotcom

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface-hub, markdown-it-py, python-dotenv, pyyaml, requests, inspect-ai, inspect-evals, openai, lighteval[accelerate,vllm], vllm, torch, transformers, accelerate, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the process of adding and managing evaluation results for Hugging Face models, making it easier to track performance and integrate with leaderboards.

Core Features & Use Cases

  • Extract from README: Parses evaluation tables directly from model README files.
  • Import from Artificial Analysis: Fetches benchmark scores from the Artificial Analysis API.
  • Run Custom Evaluations: Supports running evaluations using vLLM or lighteval for custom model testing.
  • Use Case: A researcher wants to add the latest benchmark scores from a published paper into their model card on Hugging Face. They can use this Skill to extract the table from the paper's PDF or directly import scores if the model is benchmarked on Artificial Analysis.

Quick Start

Use the hugging-face-evaluation skill to extract evaluation tables from the model's README.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add benchmark scores to a Hugging Face model card?▼

To add benchmark scores to a Hugging Face model card, this Skill parses evaluation tables from README files and formats them using the model-index metadata structure for leaderboard integration.

How do I import evaluation results from Artificial Analysis into my model card?▼

Importing evaluation results from Artificial Analysis involves fetching benchmark scores directly from its external API. This Skill automates the extraction and formats the data for your Hugging Face model card.

Can I run custom model evaluations using vLLM and lighteval?▼

Yes, you can run custom model evaluations using vLLM and lighteval. This Skill supports various inference backends and task suites to test your models and capture the results.

Does this Skill support the model-index metadata format for leaderboard integration?▼

Yes, the Skill supports the model-index metadata format for leaderboard integration. It structures extracted evaluation data to ensure compatibility with Hugging Face leaderboards.

What is the best way to extract evaluation tables from a model README?▼

The best way to extract evaluation tables from a model README is to parse the markdown structure directly. This Skill automates that extraction process to capture existing benchmark data.

Do I need Python and PyTorch dependencies to run Hugging Face model evaluations?▼

Yes, running Hugging Face model evaluations requires Python, PyTorch, and related libraries like transformers and accelerate. The Skill uses these dependencies to execute custom evaluations via vLLM.

Related Skills

Explore More Software Engineering Skills→
NagyViktNagyVikt
2

huggingface-vision-trainer

Fine-tune computer vision models on Hugging Face Jobs infrastructure.

Community
Advanced
ISAQQSAIISAQQSAI

hugging-face-evaluation-manager

Automate evaluation result updates in Hugging Face model cards.

Community
Advanced
projectedanxprojectedanx
1

evaluating-llms-harness

Benchmark Large Language Models across academic and reasoning datasets.

Community
Advanced
BlackRoad-OS-IncBlackRoad-OS-Inc
1

hugging-face-evaluation

Consolidate README benchmarks and Artificial Analysis imports into Hugging Face model-index entries.

Official
Advanced