skills.rest
truefoundry
truefoundryOFFICIAL
Explore all skills by this author →
Website𝕏@truefoundryGitHub

llm-deploy

Deploy LLMs and ML inference servers on TrueFoundry with GPU-accelerated endpoints.

13|Updated Feb 13, 2026
Search Tags:#llm#model deployment#huggingface#vllm#nvidia nim#gpu inference#tgi
One-click install
npx skills add https://github.com/truefoundry/tfy-agent-skills --skill llm-deploy
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-deploy
Source: https://github.com/truefoundry/tfy-agent-skills/tree/main/skills/llm-deploy
Command: npx skills add https://github.com/truefoundry/tfy-agent-skills --skill llm-deploy

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the deployment of Large Language Models (LLMs) and Machine Learning (ML) inference servers, enabling users to serve models efficiently on TrueFoundry with GPU acceleration.

Core Features & Use Cases

  • Model Serving: Deploy models using frameworks like vLLM, TGI, or NVIDIA NIM.
  • GPU Acceleration: Leverages GPU resources for high-performance inference.
  • YAML Manifests: Uses tfy apply with YAML manifests for declarative deployment.
  • Use Case: Deploying a Hugging Face model like Llama 3 for real-time text generation or using vLLM for an OpenAI-compatible inference endpoint.

Quick Start

Use the llm-deploy skill to deploy the model google/gemma-2b-it using vLLM.

Frequently Asked Questions about llm-deploy

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy a Hugging Face model using vLLM for GPU inference?▼

To deploy a Hugging Face model using vLLM, use this Skill to generate a YAML manifest and apply it with the `tfy apply` command. It configures GPU-accelerated inference endpoints with production-ready defaults and health probes.

What is the best way to serve LLMs on TrueFoundry with GPU acceleration?▼

The best way to serve LLMs on TrueFoundry is by using declarative YAML manifests with the `tfy apply` command. This Skill configures GPU resources, model caching, and health probes for production-ready language model serving.

Can I use NVIDIA NIM or TGI frameworks to deploy LLMs with this approach?▼

Yes, you can use NVIDIA NIM and TGI frameworks to deploy LLMs. This Skill supports deploying ML inference servers using vLLM, TGI, and NVIDIA NIM via YAML manifests applied through the `tfy apply` command.

Does deploying LLMs with vLLM provide an OpenAI-compatible inference endpoint?▼

Yes, deploying LLMs with vLLM provides an OpenAI-compatible inference endpoint. This allows you to serve Hugging Face models like Llama 3 for real-time text generation through a standardized API interface.

How are model caching and health probes handled during LLM deployment?▼

Model caching and health probes are handled automatically through production-ready defaults in the generated YAML manifests. This ensures reliable GPU-accelerated inference server operations when applied via `tfy apply`.

Related Skills

Explore More Software Engineering Skills→
truefoundrytruefoundry
13

workspaces

List TrueFoundry workspaces and clusters with FQNs and connectivity status.

Official
Basic
hsliuustc0106hsliuustc0106
84

vllm-omni-api

Bridge client applications with the vLLM-Omni API using OpenAI-compatible endpoints.

Community
Intermediate
heysuhasheysuhas
2

tensorrt-llm

Optimize large language model inference with NVIDIA TensorRT-LLM on GPU hardware.

Community
Advanced
BagelHoleBagelHole
46

model-serving-kubernetes

Deploy and manage ML models on Kubernetes with KServe and Triton.

Community
Advanced