skills.rest
cpich3g
cpich3gCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

hugging-face-evaluation

Extract evaluation tables from READMEs and update Hugging Face model-index metadata.

Updated Feb 25, 2026
Search Tags:#evaluation#huggingface#table-extraction#vllm#hf-jobs#model-card#artificial-analysis
One-click install
npx skills add https://github.com/cpich3g/ai-cost-optimiser --skill hugging-face-evaluation-cpich3g
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/cpich3g/ai-cost-optimiser/tree/main/.github/skills/hugging-face-evaluation
Command: npx skills add https://github.com/cpich3g/ai-cost-optimiser --skill hugging-face-evaluation-cpich3g

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface-hub, markdown-it-py, python-dotenv, pyyaml, requests, inspect-ai, inspect-evals, openai, lighteval[accelerate,vllm], vllm, torch, transformers, accelerate, and includes scripts (resource) components.

What problem does it solve?

Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval. Works with the model-index metadata format.

Core Features & Use Cases

  • Inspect and extract evaluation tables from model READMEs and update the model-index.
  • Import benchmark scores from Artificial Analysis API and merge them with existing results.
  • Run custom evaluations with vLLM, lighteval, and inspect-ai backends on Hugging Face Jobs.
  • Manage model-index entries safely, including merging new results and creating PRs for external repos.

Quick Start

Parse evaluation tables from a model README and push the results into the model-index.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add evaluation results to a Hugging Face model card?▼

You can update Hugging Face model cards by extracting evaluation tables from READMEs and importing scores from the Artificial Analysis API. The tool parses markdown content and commits YAML model-index updates directly to the repository.

Can I import benchmark scores from Artificial Analysis into my model-index?▼

Yes, you can import benchmark scores from the Artificial Analysis API and merge them with existing results in your model-index. This allows you to consolidate external evaluation data into your Hugging Face model card metadata safely.

How do I run custom model evaluations with vLLM and lighteval on Hugging Face Jobs?▼

You can run custom model evaluations with vLLM, lighteval, and inspect-ai backends directly on Hugging Face Jobs. This tool orchestrates the evaluation execution and automates updating the model-index with the resulting scores.

Does this tool require Python dependencies to parse markdown and update the model-index?▼

Yes, updating the model-index requires Python-based tooling including markdown-it-py, huggingface-hub, requests, and PyYAML. These dependencies are necessary to read, parse, and commit YAML metadata updates to Hugging Face repositories.

What is the best way to manage model-index entries for external repos?▼

The best way to manage model-index entries for external repos is to merge new results safely and create pull requests. This ensures that evaluation updates are reviewed before being committed to the target Hugging Face repository.

Related Skills

Explore More Data & Analytics Skills→
aisa-groupaisa-group
90

hugging-face-evaluation-manager

Manage structured evaluation results in Hugging Face model cards.

Official
Advanced
ISAQQSAIISAQQSAI

hugging-face-evaluation-manager

Automate evaluation result updates in Hugging Face model cards.

Community
Advanced
FacuMFacuM

hugging-face-evaluation

Extract evaluation tables from Hugging Face model README files.

Community
Advanced
POWERFULMOVESPOWERFULMOVES
1

hugging-face-evaluation-manager

Extract evaluation tables from READMEs and import benchmark scores from the Artificial Analysis API.

Community
Advanced