hugging-face-evaluation-manager

You can update Hugging Face model cards by extracting README markdown evaluation tables or importing scores from Artificial Analysis, then converting them into model-index YAML metadata. This integrates benchmark results directly into the HF Hub for leaderboard and Papers with Code compatibility.

Converting markdown evaluation tables into model-index YAML for Hugging Face involves parsing the README tables and generating compliant metadata. This allows the evaluation metrics to be recognized by Hugging Face leaderboards and Papers with Code.

Yes, you can run inspect-ai and vLLM or lighteval evaluations through Hugging Face Jobs using uv. This allows you to submit evaluation scripts for GPU throughput and automatically integrate the results into your model card maintenance workflow.

Yes, importing scores from Artificial Analysis and writing to the Hugging Face Hub requires integrating HF_TOKEN for Hub writes and AA_API_KEY for Artificial Analysis access. These environment variables authenticate the automated metadata updates.

The best way to automate Hugging Face model card maintenance is by using a workflow that extracts benchmark tables, generates model-index YAML, and submits inspect-ai or vLLM evaluation scripts through HF Jobs. This handles inspection, validation, and job-based runs automatically.

Yes, updating model-index metadata works with both local vLLM or lighteval runs and remote HF Jobs. The workflow parses the resulting evaluation metrics and merges them into existing model card metadata regardless of the execution environment.