hugging-face-jobs

Run arbitrary Python workloads on Hugging Face Jobs with UV or Docker and secure token authentication.

36.9k|6.1k|Updated Oct 21, 2024
One-click install
npx skills add https://github.com/patchy631/ai-engineering-hub --skill hugging-face-jobs-patchy631
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: hugging-face-jobs
Source: https://github.com/patchy631/ai-engineering-hub/tree/main/hugging-face-skills/skills/hugging-face-jobs
Command: npx skills add https://github.com/patchy631/ai-engineering-hub --skill hugging-face-jobs-patchy631

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires datasets, flashinfer-python, huggingface-hub[hf_transfer], hf-xet>=1.1.7, torch, transformers, vllm>=0.8.5, numpy, tqdm, scikit-learn, polars>=1.31.0, huggingface-hub, ascii-graph, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill enables running arbitrary Python workloads on Hugging Face Jobs infrastructure, removing the burden of local setup and enabling scalable experimentation.

Core Features & Use Cases

  • UV inline scripts & Docker-based jobs: orchestrate Python workloads with inline dependencies or containerized environments, across CPU and GPU flavors.
  • Hardware selection & cost awareness: guidelines for choosing flavors and estimating runtime costs for batches, training, or experiments.
  • Secure authentication & persistence: integrated HF_TOKEN handling via secrets with secure persistence of results to the Hub or external storage.
  • Scheduling, monitoring & reproducibility: manage job lifecycles, logs, and reproducible dataset/model/dataset-card pushes.

Quick Start

Use the skill to submit a small UV/Docker job and persist the results to the Hugging Face Hub with secure token handling.

Frequently Asked Questions about hugging-face-jobs

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run Python workloads on Hugging Face Jobs infrastructure?

To run Python workloads on Hugging Face Jobs, you can orchestrate tasks using either UV inline scripts or Docker-based containers across CPU and GPU flavors. This removes local setup burdens and enables scalable experimentation directly on the Hub's cloud compute.

Do I need an HF_TOKEN to submit jobs to Hugging Face Jobs?

Yes, a valid HF_TOKEN is required to submit jobs to Hugging Face Jobs. The skill integrates secure token handling via secrets to ensure authentication and secure persistence of results to the Hub or external storage.

What's the best way to choose hardware flavors for Hugging Face Jobs?

Choosing hardware flavors for Hugging Face Jobs involves using the skill's cost awareness guidelines to estimate runtime costs. You can select appropriate CPU or GPU flavors based on the specific requirements of your batch processing, training, or experiment workloads.

Can I use Docker containers for Hugging Face Jobs?

Yes, you can use Docker-based jobs to run containerized environments on Hugging Face Jobs. This allows you to execute arbitrary Python workloads with specific dependencies across various hardware configurations.

How do I persist results and models to the Hugging Face Hub after a job?

You can persist results to the Hugging Face Hub by using the skill's secure persistence features, which manage reproducible dataset and model card pushes. This ensures your job outputs are stored safely using token-based authentication.

How does scheduling and monitoring work for Hugging Face Jobs?

Scheduling and monitoring for Hugging Face Jobs involves managing the job lifecycle, accessing logs, and ensuring reproducibility. The skill provides guidelines to track your submitted UV or Docker workloads throughout their execution.