skills.rest
erenisiklar
erenisiklarCOMMUNITY
Explore all skills by this author →
GitHub

hugging-face-evaluation

Extract and update Hugging Face model card evaluation results from README tables and benchmarks.

Updated Mar 27, 2026
Search Tags:#automation#evaluation#benchmark#huggingface#model-index#readme-tables#aa-import
One-click install
npx skills add https://github.com/erenisiklar/Pusula --skill hugging-face-evaluation-erenisiklar
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/erenisiklar/Pusula/tree/main/.claude/skills/hugging-face-evaluation
Command: npx skills add https://github.com/erenisiklar/Pusula --skill hugging-face-evaluation-erenisiklar

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface-hub, markdown-it-py, python-dotenv, pyyaml, requests, and includes scripts (resource) components.

What problem does it solve?

Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval. Works with the model-index metadata format.

Core Features & Use Cases

  • Extracting evaluation tables from READMEs and updating model-index entries
  • Importing benchmark data from Artificial Analysis and merging with existing results
  • Running custom evaluations with vLLM or lighteval to generate new metrics
  • Integrating with Hugging Face model cards, Jobs, and Papers with Code specifications

Quick Start

Run uv run scripts/evaluation_manager.py extract-readme --repo-id "username/model" to pull evaluation tables from a README.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract evaluation tables from a Hugging Face model README?▼

Extract evaluation tables from a Hugging Face model README by running the extraction script with the target repository ID. This aggregates README-based benchmark tables and updates the model-index metadata entries automatically.

Can I import Artificial Analysis benchmark scores into Hugging Face model cards?▼

Yes, you can import Artificial Analysis benchmark scores into Hugging Face model cards. The tool uses the AA_API_KEY to fetch external benchmark data and merges it with existing evaluation results in the model-index metadata.

Does this tool support running custom model evaluations with vLLM or lighteval?▼

Yes, it supports running custom model evaluations with vLLM or lighteval to generate new metrics. These custom evaluation results are then integrated into the Hugging Face model card metadata format.

What dependencies are required to update Hugging Face model card evaluation results?▼

Updating Hugging Face model card evaluation results requires the huggingface_hub, markdown-it-py, python-dotenv, pyyaml, and requests dependencies. You also need an HF_TOKEN for authentication and PR-based updates.

What is the best way to merge external benchmarks with existing Hugging Face model-index metadata?▼

The best way to merge external benchmarks with existing Hugging Face model-index metadata is using an automated aggregation script. It parses README tables, imports Artificial Analysis data, and applies PR-based updates to the model card.

Related Skills

Explore More Data & Analytics Skills→
BlackRoad-OS-IncBlackRoad-OS-Inc
1

hugging-face-evaluation

Consolidate README benchmarks and Artificial Analysis imports into Hugging Face model-index entries.

Official
Advanced
ISAQQSAIISAQQSAI

hugging-face-evaluation-manager

Automate evaluation result updates in Hugging Face model cards.

Community
Advanced
brivarobrivaro

hugging-face-evaluation

Manage Hugging Face model evaluation results from READMEs, Artificial Analysis, and custom benchmarks.

Community
Advanced
Dianshu-LiaoDianshu-Liao
2

hugging-face-evaluation-manager

Extract README evaluation tables into Hugging Face model-index YAML.

Community
Advanced