Download and upload HuggingFace Hub models and datasets with authentication.
npx skills add https://github.com/jayll1303/AIEKit --skill hf-hub-datasets
Please help me install this Agent Skill. Skill: hf-hub-datasets Source: https://github.com/jayll1303/AIEKit/tree/main/.kiro/skills/hf-hub-datasets Command: npx skills add https://github.com/jayll1303/AIEKit --skill hf-hub-datasets
Eliminates friction when transferring, accessing, and managing models and datasets on the HuggingFace Hub by providing clear patterns for authentication, partial downloads, uploads, and large-dataset workflows so teams can move artifacts reliably between local, CI, and Hub storage.
Use the hf-hub-datasets skill to download a model snapshot and stream a large dataset for on-the-fly processing.
High-intent search queries and answers about installing and using this skill.
You can download a specific model revision from HuggingFace Hub using snapshot_download with the revision parameter. This fetches exact repository states, ensuring reproducible CI or training workflows by targeting specific commits or tags.
Yes, you can stream large HuggingFace datasets for single-pass preprocessing using iterable datasets. This memory-mapped loading approach prevents out-of-memory errors by processing data on-the-fly without downloading the entire dataset to local storage.
The best way to upload a processed dataset to HuggingFace Hub is using push_to_hub or upload_folder. These methods manage repository creation, handle Git LFS tracking for large files, and configure visibility settings for publishing artifacts.
To download only specific files from a HuggingFace model repository, use snapshot_download or hf_hub_download with allow_patterns and ignore_patterns. This partial download mechanism saves bandwidth by excluding unneeded weights or assets.
Yes, accessing gated or private HuggingFace repositories requires an HF_TOKEN for authentication. You must configure login patterns and ensure your token has the appropriate organization or private repository permissions before downloading assets.
Git LFS matters when pushing large datasets to HuggingFace Hub because it manages binary file storage outside standard Git limits. Handling LFS correctly during push_to_hub prevents upload failures and ensures repository integrity for multi-gigabyte artifacts.
JuanMS20Configure GitHub authentication for Hermes Agent using git, gh CLI, or SSH credentials.
obraValidate external data with type, range, and format checks.
microsoft-foundryAutomate end-to-end Azure AI Foundry fine-tuning workflows with dataset prep, training, evaluation, and deployment.
ArdurAIProvide expert-level Git commands and workflows for version control.
