hugging-face-evaluation

Update Hugging Face model cards with evaluation results from README tables and workflows.

4|Updated Feb 26, 2026

Search Tags:#automation #evaluation #benchmark #huggingface #model-index #model-card

Ready to run

npx skills add https://github.com/oabdelmaksoud/Openclaw-skills-Compilations --skill hugging-face-evaluation-oabdelmaksoud

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/oabdelmaksoud/Openclaw-skills-Compilations/tree/main/hugging-face-evaluation
Command: npx skills add https://github.com/oabdelmaksoud/Openclaw-skills-Compilations --skill hugging-face-evaluation-oabdelmaksoud

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface_hub, markdown-it-py, python-dotenv, pyyaml, requests, and includes scripts (resource) components.

What problem does it solve?

Automates updating Hugging Face model cards with evaluation results from README tables, Artificial Analysis imports, and vLLM/lighteval workflows.

Core Features & Use Cases

Extract evaluation tables from README and convert to model-index YAML
Import Artificial Analysis benchmarks into model cards
Run custom evaluations with vLLM/accelerate or inspect-ai via HF Jobs
Merge updates into existing model-index entries with validation and PR support
Track and compare benchmarks across models within a repository

Quick Start

Run uv run scripts/evaluation_manager.py --repo-id 'username/model' to extract evaluations from the README and update the model-index.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I update a Hugging Face model card with evaluation results from README tables?▼

To update a Hugging Face model card, run the evaluation manager script on your repository ID. It parses Markdown tables using markdown-it-py, converts them to model-index YAML, and safely merges the results into your existing model card.

Can I import Artificial Analysis benchmarks into my Hugging Face model card?▼

Yes, you can import Artificial Analysis benchmarks into your model card. The tool requires the AA_API_KEY environment variable to fetch benchmark data and automatically updates the model-index entries within your Hugging Face repository.

How do I run custom vLLM or lighteval evaluations via Hugging Face infrastructure?▼

You can run custom vLLM or lighteval evaluations by executing them as Hugging Face Jobs. The tool integrates with vLLM, accelerate, and inspect-ai workflows to execute evaluation jobs and merge the output results back into your model-index.

What dependencies and tokens are required to automate Hugging Face model evaluations?▼

Automating Hugging Face model evaluations requires the HF_TOKEN and AA_API_KEY environment variables, plus Python dependencies including huggingface_hub, markdown-it-py, python-dotenv, pyyaml, and requests to parse tables and manage the model-index schema.

Does updating the model-index with new evaluation results overwrite my existing benchmark data?▼

No, updating the model-index does not overwrite existing benchmark data. The tool safely merges new evaluation results into your existing model-index entries with validation, and it supports submitting changes via pull requests for review.