hugging-face-evaluation

To attach evaluation results to a Hugging Face model card, the Skill aggregates README-derived tables, Artificial Analysis benchmark imports, and custom model evaluations into structured model-index data. It enforces model-index compatibility for reproducible benchmarking.

Extracting benchmark tables from a README and pushing them to model-index involves parsing the markdown content and aggregating the metrics. The Skill uses markdown-it-py to process README files and structures the extracted evaluation data for Hugging Face model cards.

Yes, you can run custom evaluations with vLLM or lighteval and update model-index. The Skill supports local and remote execution via uv-based workflows across Hugging Face Jobs, integrating the custom evaluation results directly into the model card.

Yes, a Hugging Face token is required to manage model card evaluations. The Skill enforces HF token handling using python-dotenv to securely load credentials, ensuring safe dependency management and reproducible benchmarking across remote and local environments.

The best way to automate benchmarking on Hugging Face Jobs is by integrating uv.run workflows. The Skill orchestrates extraction, ingestion, and execution of AI benchmark imports and custom evaluations, ensuring model-index compatibility and reproducible results.

Model card evaluations may not update correctly due to model-index compatibility issues or missing dependencies. The Skill enforces safe dependency management and structured data aggregation from READMEs and benchmarks to ensure reproducible and correctly formatted updates.