skills.rest
brivaro
brivaroCOMMUNITY
Explore all skills by this author →
GitHub

hugging-face-evaluation

Manage Hugging Face model evaluation results from READMEs, Artificial Analysis, and custom benchmarks.

Updated Mar 8, 2026
Search Tags:#benchmarking#evaluation#huggingface#model-index#leaderboards#model cards
One-click install
npx skills add https://github.com/brivaro/brivaro-ai-wizard --skill hugging-face-evaluation-brivaro
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/brivaro/brivaro-ai-wizard/tree/main/skills/hugging-face-evaluation
Command: npx skills add https://github.com/brivaro/brivaro-ai-wizard --skill hugging-face-evaluation-brivaro

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface-hub, markdown-it-py, python-dotenv, pyyaml, requests, inspect-ai, inspect-evals, openai, lighteval[accelerate,vllm], vllm, torch, transformers, accelerate, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill streamlines the process of adding and managing evaluation results for Hugging Face models, ensuring consistency and compatibility with leaderboards and model indexing standards.

Core Features & Use Cases

  • Extract from README: Automatically parse evaluation tables directly from model README files.
  • Import from Artificial Analysis: Fetch benchmark scores directly from the Artificial Analysis API.
  • Run Custom Evaluations: Execute model evaluations using vLLM or lighteval for custom model testing.
  • Use Case: A researcher wants to add new benchmark scores to their model's card. They can use this Skill to either parse a table from their updated README or import the latest scores from Artificial Analysis, automatically creating a pull request for review.

Quick Start

Use the hugging-face-evaluation skill to inspect evaluation tables in the README of the 'username/model-name' repository.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add evaluation results to a Hugging Face model card?▼

To add evaluation results to a Hugging Face model card, you can automatically parse evaluation tables from README files or import benchmark scores directly from the Artificial Analysis API, ensuring compatibility with model indexing standards.

How do I run custom model evaluations using vLLM and lighteval?▼

You can run custom model evaluations by executing benchmarks with vLLM or lighteval, enabling tailored model testing and integrating results into model-index metadata for leaderboards.

Can I import benchmark scores from Artificial Analysis into Hugging Face?▼

Yes, you can fetch benchmark scores directly from the Artificial Analysis API and import them into your Hugging Face model repository, automatically generating a pull request for review.

Does this approach support model-index metadata for leaderboard compatibility?▼

Yes, managing Hugging Face model evaluations supports model-index metadata formatting, ensuring your extracted or custom-run benchmark results maintain strict compatibility with leaderboards.

What is the best way to parse evaluation tables from a Hugging Face README?▼

The best way to parse evaluation tables from a Hugging Face README is using automated extraction tools, streamlining the addition of benchmark scores to model cards and maintaining consistency across repositories.

Related Skills

Explore More Software Engineering Skills→
zechenzhangAGIzechenzhangAGI
11.5k

model-pruning

Prune large language models using one-shot methods like Wanda and SparseGPT.

Community
Intermediate
huggingfacehuggingface
10.9k

hugging-face-evaluation

Convert README evaluation tables into Hugging Face model-index format.

Official
Advanced
thinking-machines-labthinking-machines-lab
4.0k

evals

Evaluate language model training with inline evaluators and Inspect AI benchmarks.

Official
Intermediate
niznik-devniznik-dev
16

create-inspect-task

Create custom inspect-ai evaluation tasks with experiment-guided and standalone modes.

Community
Intermediate