hugging-face-evaluation-manager

Automating Hugging Face model card evaluation reporting involves extracting benchmark tables from model READMEs and converting them into model-index YAML. This Skill aggregates those results and pushes the updated model-index directly to your Hugging Face repository.

Yes, you can import Artificial Analysis scores into a Hugging Face model-index. By using an optional AA_API_KEY, the Skill imports external benchmark scores and merges them with your existing evaluation results while preserving source provenance.

Yes, an HF_TOKEN is required to authenticate and push updated model-index data to Hugging Face. The token enables automated PR-based contributions to your model repositories during the evaluation reporting workflow.

The best way to merge external benchmark results into Hugging Face model cards is to aggregate them into model-index YAML. This Skill extracts README benchmarks, imports Artificial Analysis data, and manages PR-based updates to the hub.

Yes, this evaluation manager supports running evaluations using vLLM, lighteval, and inspect-ai workflows. It aggregates the resulting benchmark scores and attributes their sources to maintain provenance within the Hugging Face model-index.

Hugging Face model-index updates may not reflect sources if extraction or ingestion fails. Ensure dependencies like huggingface-hub, markdown-it-py, and pyyaml are installed, and that your HF_TOKEN and AA_API_KEY are correctly configured for data merging.