skills.rest
ScientiaCapital
ScientiaCapitalCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

model-deployment

Export fine-tuned models to GGUF and deploy with vLLM or Ollama.

1|1|Updated Sep 21, 2025
Search Tags:#Docker#GGUF#model deployment#vLLM#HuggingFace Hub#production serving
One-click install
npx skills add https://github.com/ScientiaCapital/unsloth-mcp-server --skill model-deployment-scientiacapital
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-deployment
Source: https://github.com/ScientiaCapital/unsloth-mcp-server/tree/main/.claude/skills/model-deployment
Command: npx skills add https://github.com/ScientiaCapital/unsloth-mcp-server --skill model-deployment-scientiacapital

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Export and deploy fine-tuned models to production.

Core Features & Use Cases

  • GGUF export: export fine-tuned models for local or edge inference with llama.cpp/Ollama.
  • Production deployment: set up and run high-throughput serving with vLLM, Ollama, or Docker-based deployments.
  • Hub sharing & versioning: publish and version models on HuggingFace Hub for collaboration and reuse.
  • Use Case: A medical domain team finishes fine-tuning a model and deploys it to a vLLM server behind a load balancer for 24/7 API access.

Quick Start

  1. Export the fine-tuned model to GGUF, e.g., model.save_pretrained_gguf('./gguf_output', tokenizer, quantization_method='q4_k_m').
  2. Deploy with Ollama or vLLM: for Ollama, ollama create my-model -f Modelfile; for vLLM, start the server with the appropriate model path.
  3. Optionally push the model to HuggingFace Hub for sharing and versioning.

Frequently Asked Questions about model-deployment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I export a fine-tuned model to production formats like GGUF?▼

Export fine-tuned models to GGUF format using model.save_pretrained_gguf() with your tokenizer and quantization method, enabling deployment via llama.cpp, Ollama, or edge inference platforms.

What's the best way to deploy a model with vLLM for high-throughput serving?▼

Deploy models to vLLM by specifying the model path when starting the server, enabling high-throughput API access. vLLM handles batching and optimization for production workloads.

Can I share and version my fine-tuned model on HuggingFace Hub?▼

Push exported models to HuggingFace Hub for collaboration, versioning, and reuse. This enables easy distribution and tracking across teams.

Do I need Docker to deploy models in production?▼

Docker-based deployment is one option for production serving. Alternatively, deploy directly with vLLM, Ollama, or llama.cpp depending on your infrastructure and throughput requirements.

What's the difference between GGUF and vLLM deployment approaches?▼

GGUF exports optimize models for local or edge inference with llama.cpp and Ollama. vLLM focuses on high-throughput API serving with batching for production backends.

Related Skills

Explore More Software Engineering Skills→
EstomEstom

fine-tuning-expert

Fine-tune large language models with LoRA, QLoRA, and PEFT workflows.

Community
Advanced
nirukk52nirukk52
1

dev-log-monitoring

Monitor Encore.ts and SvelteKit logs to diagnose drift and verify run events.

Community
Advanced
RachasumanthRachasumanth

inference-deploy

Deploy trained machine learning models for inference using vLLM, TGI, Ollama, and llama.cpp.

Community
Advanced
doanchienthangdevdoanchienthangdev
4

Model Serving

Deploy machine learning models with TensorFlow Serving, TorchServe, or Triton Inference Server.

Community
Advanced