What problem does it solve?
This Skill addresses the challenge of running GPU-intensive ML workloads without the need for infrastructure management, providing on-demand GPU access and efficient deployment of ML models.
Core Features & Use Cases
- Serverless GPUs: Access to a variety of GPUs like T4, L4, A10G, L40S, A100, H100, H200, B200 on-demand.
- Python-native: Define infrastructure in Python code without YAML.
- Auto-scaling: Scale resources to zero or up to 100+ GPUs instantly.
- Sub-second Cold Starts: Rust-based infrastructure for fast container launches.
- Container Caching: Image layers cached for rapid iteration.
- Web Endpoints: Deploy functions as REST APIs with zero-downtime updates.
- Use Case: Ideal for deploying ML models as APIs, running batch jobs with automatic scaling, and prototyping ML applications quickly.
Quick Start
Install the Modal CLI and authenticate:
pip install modal
modal setup
Create a simple ML model function and deploy:
import modal
app = modal.App("hello-gpu")
@app.function(gpu="T4")
def gpu_info():
import subprocess
return subprocess.run(["nvidia-smi"], capture_output=True, text=True).stdout
@app.local_entrypoint()
def main():
print(gpu_info.remote())
