hugging-face-evaluation
OfficialTrack model-card evaluations with ease and reliability.
Authorhuggingface
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill centralizes and streamlines the process of recording, updating, and validating evaluation results for Hugging Face model cards by converting README-stored scores and API-sourced benchmarks into a standardized model-index format.
Core Features & Use Cases
- Read README evaluation tables and convert them into model-index entries for display in the model card widget.
- Import benchmark scores from Artificial Analysis and merge them with existing evaluations.
- Run custom model evaluations using vLLM or lighteval for GPU-backed workflows and then push updates via PRs or direct commits.
Quick Start
- Inspect a model README to identify tables: uv run scripts/evaluation_manager.py inspect-tables --repo-id <owner>/<model>.
- Extract a specific table to YAML: uv run scripts/evaluation_manager.py extract-readme --repo-id <owner>/<model> --table 1 --task-type "text-generation" --apply.
- Or import Artificial Analysis evaluations: AA_API_KEY=... uv run scripts/evaluation_manager.py import-aa --creator-slug <creator> --model-name <model> --repo-id <owner>/<model> --create-pr
Dependency Matrix
Required Modules
huggingface-hubmarkdown-it-pypython-dotenvpyyamlrequests
Components
scripts
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: hugging-face-evaluation Download link: https://github.com/huggingface/skills/archive/main.zip#hugging-face-evaluation Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.