skills.rest
MLT-OSS
MLT-OSSOFFICIAL
Explore all skills by this author →
WebsiteGitHub

modal-serverless-gpu

Deploy and run ML workloads on Modal's serverless GPU cloud platform.

14|5|Updated Apr 9, 2026
Search Tags:#serverless#cloud#gpu#batch processing#modal#inference#ml deployment
One-click install
npx skills add https://github.com/MLT-OSS/hermes-agent-go --skill modal-serverless-gpu-mlt-oss
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: modal-serverless-gpu
Source: https://github.com/MLT-OSS/hermes-agent-go/tree/main/skills/mlops/cloud/modal
Command: npx skills add https://github.com/MLT-OSS/hermes-agent-go --skill modal-serverless-gpu-mlt-oss

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires modal, and includes references (resource) components.

What problem does it solve? Running GPU-intensive ML workloads typically requires provisioning servers, managing infrastructure, and paying for idle compute. This Skill guides you through using Modal to run training, inference, and batch jobs on on-demand GPUs with pay-per-second pricing and automatic scaling. ## Core Features & Use Cases - Serverless GPU Execution: Run Python functions on T4, A10G, L40S, A100, H100, and B200 GPUs without managing infrastructure. - Model Deployment as APIs: Deploy ML models as auto-scaling FastAPI endpoints with dynamic batching and zero-downtime updates. - Batch & Scheduled Jobs: Fan out parallel processing with .map() and run cron-scheduled workloads. - Use Case: Deploy a text-generation model on an A10G GPU as a REST endpoint that scales to zero when idle and handles concurrent requests with model loading at container start. ## Quick Start Ask the agent to write and deploy a Modal app that runs a GPU inference function and exposes it as a web endpoint.

Frequently Asked Questions about modal-serverless-gpu

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a Python function on a GPU with Modal?▼

Install modal, run modal setup to authenticate, then decorate a function with @app.function(gpu="T4") inside a modal.App. Execute it with modal run script.py and the function runs remotely on the requested GPU.

How do I deploy an ML model as an API on Modal?▼

Define a class with @app.cls(gpu=...) and load the model in an @modal.enter() method, then expose it with @modal.fastapi_endpoint() or @modal.asgi_app(). Run modal deploy script.py for a persistent, auto-scaling deployment.

Which Modal GPU should I choose for inference vs training?▼

For inference, L40S (48GB) offers the best cost-to-performance ratio, while T4 suits budget workloads. For training large models, use A100-80GB or H100, and specify multiple GPUs like gpu="H100:4" for distributed jobs.

Why is my Modal function slow on the first request?▼

Cold starts occur when no warm container exists. Increase container_idle_timeout, set keep_warm=1, load models in @modal.enter() at container start, and cache model weights in a Modal Volume to reduce startup latency.

When should I use RunPod or Kubernetes instead of Modal?▼

Use RunPod for long-running pods with persistent state, Lambda Labs for reserved GPU instances, SkyPilot for multi-cloud cost optimization, and Kubernetes for complex multi-service architectures. Modal fits ephemeral, auto-scaling Python workloads.

Related Skills

Explore More Software Engineering Skills→
ethanolivertroyethanolivertroy
11

huggingface-backup

Upload local folders to private Hugging Face Buckets using hf CLI.

Community
Advanced
ToXMonToXMon

domino-model-endpoints

Deploys and monitors model API endpoints on Domino Data Lab via REST and Grafana dashboards.

Community
Advanced
0xtaoist0xtaoist

ln-627-observability-auditor

Audit logging, health checks, metrics, and tracing in codebases.

Community
Advanced
zainhaszainhas
2

together-gpu-clusters

Provision GPU clusters on Together AI with Kubernetes or Slurm orchestration.

Community
Advanced