skills.rest
oabdelmaksoud
oabdelmaksoudCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

hugging-face-evaluation

Update Hugging Face model cards with evaluation results from README tables and workflows.

4|Updated Feb 26, 2026
Search Tags:#automation#evaluation#benchmark#huggingface#model-index#model-card
One-click install
npx skills add https://github.com/oabdelmaksoud/Openclaw-skills-Compilations --skill hugging-face-evaluation-oabdelmaksoud
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/oabdelmaksoud/Openclaw-skills-Compilations/tree/main/hugging-face-evaluation
Command: npx skills add https://github.com/oabdelmaksoud/Openclaw-skills-Compilations --skill hugging-face-evaluation-oabdelmaksoud

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface_hub, markdown-it-py, python-dotenv, pyyaml, requests, and includes scripts (resource) components.

What problem does it solve?

Automates updating Hugging Face model cards with evaluation results from README tables, Artificial Analysis imports, and vLLM/lighteval workflows.

Core Features & Use Cases

  • Extract evaluation tables from README and convert to model-index YAML
  • Import Artificial Analysis benchmarks into model cards
  • Run custom evaluations with vLLM/accelerate or inspect-ai via HF Jobs
  • Merge updates into existing model-index entries with validation and PR support
  • Track and compare benchmarks across models within a repository

Quick Start

Run uv run scripts/evaluation_manager.py --repo-id 'username/model' to extract evaluations from the README and update the model-index.

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I update a Hugging Face model card with evaluation results from README tables?▼

To update a Hugging Face model card, run the evaluation manager script on your repository ID. It parses Markdown tables using markdown-it-py, converts them to model-index YAML, and safely merges the results into your existing model card.

Can I import Artificial Analysis benchmarks into my Hugging Face model card?▼

Yes, you can import Artificial Analysis benchmarks into your model card. The tool requires the AA_API_KEY environment variable to fetch benchmark data and automatically updates the model-index entries within your Hugging Face repository.

How do I run custom vLLM or lighteval evaluations via Hugging Face infrastructure?▼

You can run custom vLLM or lighteval evaluations by executing them as Hugging Face Jobs. The tool integrates with vLLM, accelerate, and inspect-ai workflows to execute evaluation jobs and merge the output results back into your model-index.

What dependencies and tokens are required to automate Hugging Face model evaluations?▼

Automating Hugging Face model evaluations requires the HF_TOKEN and AA_API_KEY environment variables, plus Python dependencies including huggingface_hub, markdown-it-py, python-dotenv, pyyaml, and requests to parse tables and manage the model-index schema.

Does updating the model-index with new evaluation results overwrite my existing benchmark data?▼

No, updating the model-index does not overwrite existing benchmark data. The tool safely merges new evaluation results into your existing model-index entries with validation, and it supports submitting changes via pull requests for review.

Related Skills

Explore More Data & Analytics Skills→
erenisiklarerenisiklar

hugging-face-evaluation

Extract and update Hugging Face model card evaluation results from README tables and benchmarks.

Community
Advanced
Himanshu040604Himanshu040604

hugging-face-evaluation

Extract evaluation tables from Hugging Face model README files.

Community
Advanced
FacuMFacuM

hugging-face-evaluation

Extract evaluation tables from Hugging Face model README files.

Community
Advanced
Dianshu-LiaoDianshu-Liao
2

hugging-face-evaluation-manager

Extract README evaluation tables into Hugging Face model-index YAML.

Community
Advanced