skills.rest
jfriisj
jfriisjCOMMUNITY
Explore all skills by this author →
GitHub

container-llm-cuda-optimization

Build GPU-accelerated Docker containers for LLMs with multi-stage CUDA builds.

Updated Mar 20, 2026
Search Tags:#llm#docker#gpu#pytorch#cuda#image-optimization#multistage-build
One-click install
npx skills add https://github.com/jfriisj/coding-agents --skill container-llm-cuda-optimization
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: container-llm-cuda-optimization
Source: https://github.com/jfriisj/coding-agents/tree/main/context-ligt-workflow/skills/container-llm-cuda-optimization
Command: npx skills add https://github.com/jfriisj/coding-agents --skill container-llm-cuda-optimization

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Expert guidance for building highly optimized, GPU-accelerated Docker containers for LLMs and CUDA, helping prevent image bloat and ensure correct runtime vs devel configurations across development to production.

Core Features & Use Cases

  • Nvidia Base Image Strategy: Use the correct base image tier (base, runtime, devel) to minimize image size while providing required libraries.
  • Multi-Stage CUDA Architecture: Build complex CUDA extensions in a devel stage and copy built artifacts into a lean runtime stage.
  • Dependency Management: Explicitly pin PyTorch CUDA flavor via the right index URL to avoid mixing CPU and GPU wheels.
  • Weights Handling Guidance: Avoid baking 20+ GB weights into images; mount weights at runtime or download to a shared volume.

Quick Start

Apply the multi-stage CUDA Docker strategy to build a lean runtime image for deploying LLM workloads.

Frequently Asked Questions about container-llm-cuda-optimization

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a Docker image for PyTorch and CUDA without bloating the final size?▼

Use multi-stage CUDA builds to compile complex extensions in a devel stage and copy only the built artifacts into a lean runtime image. This prevents Docker image bloat while ensuring correct runtime configurations for PyTorch workloads.

What's the best way to manage large LLM model weights in Docker containers?▼

Avoid baking 20+ GB weights into Docker images. Mount model weights at runtime or download them to a shared volume. This keeps containers lightweight and ensures correct runtime versus development environments for LLMs.

Why does my PyTorch Docker container run on CPU instead of GPU?▼

Mixing CPU and GPU wheels causes PyTorch to default to CPU. Explicitly pin the PyTorch CUDA flavor via the correct index URL during dependency management to ensure GPU acceleration works correctly in Docker.

When do I need the Nvidia devel base image versus the runtime base image?▼

Use the Nvidia devel base image to compile complex CUDA extensions, then copy artifacts into a lean runtime image. Selecting the correct base image tier minimizes final image size while providing required libraries for GPU workloads.

Can I use multi-stage builds to optimize CUDA development and production environments?▼

Yes, multi-stage builds separate devel and runtime environments for CUDA workloads. Build extensions in the devel stage and copy artifacts into a lean runtime stage to ensure optimized GPU-accelerated containers from development to production.

Does this approach support deploying LLM workloads across development and production?▼

Yes, the multi-stage CUDA Docker strategy establishes correct runtime versus devel configurations for LLMs. It guides base image selection and dependency management to ensure optimized GPU acceleration across development to production.

Related Skills

Explore More Software Engineering Skills→
jayll1303jayll1303
17

docker-gpu-setup

Build GPU-enabled Docker containers with NGC base images and uv.

Community
Advanced
dralkhdralkh
74

modal

Deploy Python workloads on Modal's serverless cloud platform.

Community
Advanced
ldzhhxxldzhhxx
1

tensorrt-llm

Optimize LLM inference with NVIDIA TensorRT for production GPUs.

Community
Advanced
augustoheissaugustoheiss

agency-ai-engineer

Provide end-to-end machine learning engineering support for deploying production models.

Community
Advanced