skills.rest
Bennoo
BennooCOMMUNITY
Explore all skills by this author →
𝕏@bantoutouGitHub

hugging-face-evaluation

Automate Hugging Face model card updates with evaluation results.

4|Updated Jan 12, 2026
One-click install
npx skills add https://github.com/Bennoo/classification_experience --skill hugging-face-evaluation-bennoo
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hugging-face-evaluation
Source: https://github.com/Bennoo/classification_experience/tree/main/.claude/skills/hugging-face-evaluation
Command: npx skills add https://github.com/Bennoo/classification_experience --skill hugging-face-evaluation-bennoo

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps teams keep Hugging Face model cards up to date by automating the intake and integration of evaluation results from multiple sources.

Core Features & Use Cases

  • Extract evaluation tables from READMEs and convert them into model-index YAML for leaderboard display.
  • Import benchmark scores from Artificial Analysis and merge them with existing model cards.
  • Run custom model evaluations using vLLM or accelerate backends and publish results to model cards via HF Jobs integration.
  • Validate, preview, and push changes with PR handling to collaborative repos.

Quick Start

  1. Install dependencies and set HF_TOKEN (and AA_API_KEY if using Artificial Analysis).
  2. Extract evaluation tables from README: uv run scripts/evaluation_manager.py extract-readme --repo-id "<your-org>/<model>".
  3. Optionally import AA scores: AA_API_KEY="your-aa-key" uv run scripts/evaluation_manager.py import-aa --creator-slug "<creator>" --model-name "<model>" --repo-id "<creator>/<model>" --create-pr
  4. Review and push changes: uv run scripts/evaluation_manager.py show --repo-id "<your-org>/<model>" && uv run scripts/evaluation_manager.py validate --repo-id "<your-org>/<model>"

Frequently Asked Questions about hugging-face-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate updating Hugging Face model cards with evaluation results?▼

To automate updating Hugging Face model cards with evaluation results, use this skill to ingest, format, and publish metrics into model-index YAML. It supports extracting tables from READMEs, importing benchmarks, and pushing changes via automated PR handling.

How do I extract evaluation tables from a README and convert them to model-index YAML?▼

You can extract evaluation tables from a README and convert them to model-index YAML for leaderboard display by running the evaluation manager script. Use the 'extract-readme' command with your specific Hugging Face repo ID.

Can I import Artificial Analysis benchmark scores into my Hugging Face model card?▼

Yes, you can import Artificial Analysis benchmark scores into your Hugging Face model card. The skill merges imported AA scores with existing model cards and optionally creates pull requests using your AA_API_KEY and HF_TOKEN.

Does this skill support running custom model evaluations using vLLM?▼

Yes, this skill supports running custom model evaluations using vLLM or accelerate backends. It publishes the results directly to your Hugging Face model cards through integrated HF Jobs.

What do I need to set up before validating and pushing evaluation data to collaborative repos?▼

Before validating and pushing evaluation data to collaborative repos, you need to install dependencies and set your HF_TOKEN. If importing Artificial Analysis scores, you must also configure the AA_API_KEY environment variable.

Related Skills

luokai25luokai25

hugging-face-tool-builder

Creates and executes scripts that interact with the Hugging Face API.

Community
Advanced
lidge-junlidge-jun
5

hugging-face-evaluation

Manage Hugging Face model evaluation results from READMEs and APIs.

Community
Advanced
BlackRoad-OS-IncBlackRoad-OS-Inc
1

hugging-face-evaluation

Consolidate README benchmarks and Artificial Analysis imports into Hugging Face model-index entries.

Official
Advanced
ISAQQSAIISAQQSAI

hugging-face-evaluation-manager

Automate evaluation result updates in Hugging Face model cards.

Community
Advanced