skills.rest
AtlasOmnia
AtlasOmniaCOMMUNITY
Explore all skills by this author →
Website𝕏@AtlasOmniaGitHub

hf-model-card-research

Extract benchmark claims, download stats, and metadata from HuggingFace model cards.

115|9|Updated Aug 5, 2026
Search Tags:#api#huggingface#model cards#llm comparison#model research#benchmark extraction#download stats
One-click install
npx skills add https://github.com/AtlasOmnia/donna-starter --skill hf-model-card-research-atlasomnia
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hf-model-card-research
Source: https://github.com/AtlasOmnia/donna-starter/tree/main/skills/research/hf-model-card-research
Command: npx skills add https://github.com/AtlasOmnia/donna-starter --skill hf-model-card-research-atlasomnia

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires requests.

What problem does it solve? Comparing HuggingFace model variants requires manually visiting dozens of model cards to collect download counts, benchmark tables, and author claims, which is slow and error-prone. This Skill automates pulling structured metadata via the HuggingFace API and extracting benchmark prose from README files so variants can be compared side by side. ## Core Features & Use Cases - Structured stats via HF API: Pull downloads, likes, pipeline tags, file lists, and GGUF metadata from huggingface.co/api/models/{owner}/{repo} using a bounded Python script with per-repo error handling. - Benchmark claim extraction: Read README benchmark tables (SWE-bench, AIME, MMLU-Pro, GPQA, ARC-c, refusal rates, KL divergence) via web_extract and flag when cards only reproduce base-model numbers. - Cross-variant comparison: Compile per-variant reports and summary tables grouping models by real benchmark data, refusal-rate tests, and popularity. - Use Case: Given a list of 22 fine-tunes of a new base model, produce a verified stats file with downloads, quant file sizes, and a comparison of which variants actually publish benchmark deltas versus the base model. ## Quick Start Pull the HuggingFace stats and benchmark claims for these model repos and compile a comparison table of downloads, likes, and reported benchmarks.

Frequently Asked Questions about hf-model-card-research

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I pull download counts and likes for HuggingFace models programmatically?▼

Query the HuggingFace API endpoint https://huggingface.co/api/models/{owner}/{repo}, which returns JSON with downloads, likes, pipeline_tag, tags, and file siblings. Loop over repos with a requests.Session and per-repo exception handling so one failure does not stop the batch.

How to extract benchmark tables from HuggingFace model card READMEs?▼

Fetch the README at https://huggingface.co/{owner}/{repo}/resolve/main/README.md or use web_extract on the web-view URL in batches of up to 5 URLs. Look for SWE-bench, AIME, MMLU-Pro, GPQA, ARC-c, and refusal-rate tables, and page through cached output for truncated cards.

Why do delegated subagents fail when pulling HuggingFace API data?▼

Subagents loop on the same failing terminal call and hit the same_tool_failure_halt guardrail after four non-progressing retries, producing zero data. Run a single bounded host-side Python script with per-repo exception handling instead of delegating HF stats pulls.

Do HuggingFace fine-tune cards always report their own benchmark results?▼

No. Many fine-tune cards reproduce only the base model's official benchmarks or list no deltas at all, and many have null model-index in the API. Always flag when numbers are base-model results, pending, or absent rather than treating them as variant results.

Can I get GGUF file sizes from the HuggingFace API?▼

Yes. The API JSON includes a siblings array listing repo files and a gguf block with totalFileSize for GGUF repos. Filter sibling filenames ending in .gguf to enumerate quant files, and check the README download table for explicit per-quant sizes.

Related Skills

Explore More Education & Research Skills→
AtlasOmniaAtlasOmnia
115

hf-model-card-research

Extract benchmark claims, download stats, and metadata from HuggingFace model cards.

Community
Intermediate
yanochka11yanochka11

huggingface-best

Recommend HuggingFace models by task, benchmarks, and hardware constraints.

Community
Advanced
AscendAscend
14

perf-gate

Validates performance optimization claims on shared Ascend NPU machines using same-window A/B measurement protocols.

Official
Advanced
AnandaAnugrahHandyantoAnandaAnugrahHandyanto

evaluating-llms-harness

Benchmark LLM performance across 60+ tasks using lm-evaluation-harness.

Community
Advanced