skills.rest
xu1713
xu1713COMMUNITY
Explore all skills by this author →
GitHub

modal-serverless-gpu

Deploy ML workloads and GPU functions on Modal's serverless cloud platform.

Updated Jun 5, 2026
Search Tags:#deployment#serverless#python#cloud#gpu#modal#ml inference
One-click install
npx skills add https://github.com/xu1713/openhorse --skill modal-serverless-gpu-xu1713
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: modal-serverless-gpu
Source: https://github.com/xu1713/openhorse/tree/main/openhorse/openhorse/optional-skills/mlops/modal
Command: npx skills add https://github.com/xu1713/openhorse --skill modal-serverless-gpu-xu1713

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires modal, and includes references (resource) components.

What problem does it solve? Running GPU-intensive ML workloads typically requires provisioning servers, managing infrastructure, and paying for idle compute. This Skill guides you through using Modal to run Python functions on serverless GPUs with pay-per-second pricing and automatic scaling. ## Core Features & Use Cases - Serverless GPU Execution: Run functions on T4, A10G, L40S, A100, H100, and B200 GPUs with sub-second cold starts and scale-to-zero billing. - Model Deployment as APIs: Deploy ML models as auto-scaling FastAPI endpoints, ASGI/WSGI apps, or scheduled cron jobs with zero-downtime updates. - Batch and Parallel Processing: Fan out workloads across hundreds of containers using map/starmap and dynamic batching for GPU-efficient inference. - Use Case: Deploy a text-generation model on an A10G GPU as a REST endpoint that scales from zero to dozens of containers during traffic spikes, paying only for actual compute time. ## Quick Start Install the modal package, run modal setup to authenticate, then ask the assistant to deploy a Python function with a GPU decorator as a web endpoint on Modal.

Frequently Asked Questions about modal-serverless-gpu

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy an ML model as an API on Modal?▼

Define a Modal app with a function decorated with @modal.fastapi_endpoint() or wrap a FastAPI app with @modal.asgi_app(), then run modal deploy script.py. Modal builds the container, provisions GPUs, and exposes an HTTPS endpoint with automatic scaling.

How do I run a Python function on a GPU with Modal?▼

Decorate a function with @app.function(gpu="A10G") specifying the GPU type, then call it with .remote() or run modal run script.py. Modal provisions the GPU container on demand and bills per second of usage.

Modal vs RunPod vs SkyPilot for GPU workloads?▼

Modal suits serverless, scale-to-zero workloads with Python-defined infrastructure and sub-second cold starts. RunPod fits longer-running persistent pods, while SkyPilot handles multi-cloud orchestration and cost optimization across providers.

Which GPUs does Modal support?▼

Modal offers T4, L4, A10G, L40S, A100-40GB, A100-80GB, H100, H200, and B200 GPUs. You can request up to 8 GPUs per function, specify fallbacks like gpu=["H100", "A100"], or use gpu="any" for any available GPU.

Why is my Modal function slow on the first request?▼

Cold starts occur when no warm container exists. Mitigate by increasing container_idle_timeout, setting keep_warm=1, loading models in an @modal.enter() lifecycle hook, and caching model weights in a Modal Volume.

Why are my Modal volume changes not persisting?▼

Volume writes are not saved until you call volume.commit() after writing files. Conversely, use volume.reload() in reading functions to fetch the latest data committed by other containers.

Related Skills

Explore More Software Engineering Skills→
Ritabrata-ChakrabortyRitabrata-Chakraborty

modal

Run Python code serverlessly on cloud infrastructure with optional GPU acceleration.

Community
Advanced
jvdominojvdomino

domino-model-endpoints

Deploy and monitor model endpoints in Domino with Grafana and NVIDIA Triton support.

Community
Advanced
organvm-i-theoriaorganvm-i-theoria

ml-experiment-tracker

Track ML experiments, version datasets, and manage model registries with MLflow, W&B, and DVC.

Official
Intermediate
eddieluengeddielueng
1

lambda-labs-gpu-cloud

Provision reserved Lambda Labs GPU instances with persistent filesystems and SSH access.

Community
Intermediate