skills.rest
open-thoughts
open-thoughtsOFFICIAL
Explore all skills by this author →
WebsiteGitHub

sft-job-cleanup

Publish finished LLaMA-Factory SFT jobs to Hugging Face and clean up HPC cluster state.

276|37|Updated Dec 5, 2025
Search Tags:#supabase#cleanup#checkpoint#upload#hpc#huggingface#sft
One-click install
npx skills add https://github.com/open-thoughts/OpenThoughts-Agent --skill sft-job-cleanup
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sft-job-cleanup
Source: https://github.com/open-thoughts/OpenThoughts-Agent/tree/main/.claude/skills/sft-job-cleanup
Command: npx skills add https://github.com/open-thoughts/OpenThoughts-Agent --skill sft-job-cleanup

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Publish + clean up a finished LLaMA-Factory SFT job on a no-internet HPC cluster (Jupiter/Leonardo): cancel pending retries, drop intermediate checkpoints, HF-upload the model to its configured --hub_model_id, and free disk. Covers the 8B path (root safetensors, direct upload), the 32B/ZeRO-3 path (consolidate shards → safetensors first), the Qwen3.5 preprocessor_config copy, the don't-upload-partials policy, and the hf-upload gotchas (tmux not nohup, hf upload not -large-folder, Leonardo sbatch-tunnel not login node). Use when an SFT fine-tune finishes and needs uploading + registering, or "run the SFT cleanup checklist". Distinct from RL cleanup (rl-job-cleanup) and datagen cleanup (datagen-job-cleanup).

Core Features & Use Cases

  • End-to-end post-run cleanup: cancel retries, drop checkpoints, upload to Hugging Face, and register in the unified DB.
  • No-internet HPC workflow safety: handles upload via tmux sessions and Leonardo sbatch-tunnel patterns.
  • Policy-compliant cleanup: enforces the don't-upload-partials rule, validates tokenizer, and frees disk after successful publish.

Quick Start

Run the SFT cleanup workflow after a finished fine-tune to publish, upload, register, and free resources.

Frequently Asked Questions about sft-job-cleanup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I upload a finished LLaMA-Factory SFT model from a no-internet HPC cluster to Hugging Face?▼

To upload a finished LLaMA-Factory SFT model from a no-internet HPC cluster, use tmux sessions and the Leonardo sbatch-tunnel pattern to transfer root safetensors or consolidated ZeRO-3 shards to your configured Hugging Face hub.

What is the correct way to clean up intermediate checkpoints after an SFT job on an HPC cluster?▼

Proper SFT job cleanup involves canceling pending retries, removing intermediate checkpoints, uploading the final model to Hugging Face, registering it in Supabase, and freeing disk space to ensure no partial files are uploaded.

How do I consolidate ZeRO-3 shards for a 32B model before uploading to Hugging Face?▼

Consolidating ZeRO-3 shards for a 32B model requires merging the distributed checkpoint shards into unified safetensors before initiating the Hugging Face upload, ensuring the final model weights are properly assembled.

Does the SFT cleanup workflow support both 8B and 32B model paths?▼

Yes, the SFT cleanup workflow supports the 8B path with direct root safetensors upload and the 32B path with ZeRO-3 shard consolidation, covering the Qwen3.5 preprocessor_config copy for both model scales.

Why should I avoid using nohup and hf upload-large-folder for HPC model uploads?▼

Avoid nohup and hf upload-large-folder because the SFT cleanup policy requires using tmux for persistent sessions and the standard hf upload command to prevent partial uploads and ensure policy-compliant Hugging Face transfers.

What limitations exist when publishing SFT models on Leonardo HPC clusters?▼

Leonardo HPC clusters have no-internet constraints requiring sbatch-tunnel uploads instead of login node transfers, enforcing a strict don't-upload-partials policy and requiring Supabase registration to fully complete the publish cycle.

Related Skills

Explore More Software Engineering Skills→
open-thoughtsopen-thoughts
276

sft-launch-jupiter

Launch SFT experiments on Jupiter via hpc.launch with LLaMA-Factory.

Official
Advanced
zangjeicyzangjeicy

huggingface-hub

Manage Hugging Face Hub repositories, datasets, and spaces via CLI.

Community
Intermediate
open-thoughtsopen-thoughts
276

datagen-job-cleanup

Upload verified datagen traces to HF and free disk space.

Official
Advanced
peteromalletpeteromallet
97

huggingface-hub

Automate Hugging Face Hub asset management using the hf CLI.

Community
Advanced