hugging-face-evaluation

Manage Hugging Face model evaluation results from READMEs, Artificial Analysis, and custom benchmarks.

Updated Mar 8, 2026

Search Tags:#benchmarking #evaluation #huggingface #model-index #leaderboards #model cards

Ready to run

npx skills add https://github.com/brivaro/brivaro-ai-wizard --skill hugging-face-evaluation-brivaro

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/brivaro/brivaro-ai-wizard/tree/main/skills/hugging-face-evaluation
Command: npx skills add https://github.com/brivaro/brivaro-ai-wizard --skill hugging-face-evaluation-brivaro

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface-hub, markdown-it-py, python-dotenv, pyyaml, requests, inspect-ai, inspect-evals, openai, lighteval[accelerate,vllm], vllm, torch, transformers, accelerate, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill streamlines the process of adding and managing evaluation results for Hugging Face models, ensuring consistency and compatibility with leaderboards and model indexing standards.

Core Features & Use Cases

Extract from README: Automatically parse evaluation tables directly from model README files.
Import from Artificial Analysis: Fetch benchmark scores directly from the Artificial Analysis API.
Run Custom Evaluations: Execute model evaluations using vLLM or lighteval for custom model testing.
Use Case: A researcher wants to add new benchmark scores to their model's card. They can use this Skill to either parse a table from their updated README or import the latest scores from Artificial Analysis, automatically creating a pull request for review.

Quick Start

Use the hugging-face-evaluation skill to inspect evaluation tables in the README of the 'username/model-name' repository.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I add evaluation results to a Hugging Face model card?▼

To add evaluation results to a Hugging Face model card, you can automatically parse evaluation tables from README files or import benchmark scores directly from the Artificial Analysis API, ensuring compatibility with model indexing standards.

How do I run custom model evaluations using vLLM and lighteval?▼

You can run custom model evaluations by executing benchmarks with vLLM or lighteval, enabling tailored model testing and integrating results into model-index metadata for leaderboards.

Can I import benchmark scores from Artificial Analysis into Hugging Face?▼

Yes, you can fetch benchmark scores directly from the Artificial Analysis API and import them into your Hugging Face model repository, automatically generating a pull request for review.

Does this approach support model-index metadata for leaderboard compatibility?▼

Yes, managing Hugging Face model evaluations supports model-index metadata formatting, ensuring your extracted or custom-run benchmark results maintain strict compatibility with leaderboards.

What is the best way to parse evaluation tables from a Hugging Face README?▼

The best way to parse evaluation tables from a Hugging Face README is using automated extraction tools, streamlining the addition of benchmark scores to model cards and maintaining consistency across repositories.