gpu-server-management

Configure NVIDIA GPU servers with drivers, CUDA, and MIG partitioning.

46|4|Updated Jan 27, 2026

Search Tags:#mlops #ai #llm #gpu #server management #cuda #nvidia

Ready to run

npx skills add https://github.com/BagelHole/DevOps-Security-Agent-Skills --skill gpu-server-management

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: gpu-server-management
Source: https://github.com/BagelHole/DevOps-Security-Agent-Skills/tree/main/infrastructure/servers/gpu-server-management
Command: npx skills add https://github.com/BagelHole/DevOps-Security-Agent-Skills --skill gpu-server-management

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the complex process of setting up and managing NVIDIA GPU servers, ensuring optimal performance for AI workloads like LLM inference and training.

Core Features & Use Cases

Driver & Toolkit Installation: Automates the installation of NVIDIA drivers, CUDA, and Container Toolkit.
Performance Tuning: Configures persistence mode, power limits, and PCIe settings for efficiency.
Monitoring & Diagnostics: Integrates tools like nvidia-smi and DCGM for health checks and performance tracking.
MIG Partitioning: Enables slicing of high-end GPUs (A100/H100) for multi-tenant environments.
Use Case: Quickly deploy a new server for LLM training by installing all necessary NVIDIA software, configuring optimal settings, and verifying GPU health with a single command.

Quick Start

Install NVIDIA drivers, CUDA toolkit, and the NVIDIA Container Toolkit on Ubuntu 22.04.

Frequently Asked Questions about gpu-server-management

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I set up an NVIDIA GPU server for LLM training on Ubuntu?▼

To set up an NVIDIA GPU server for LLM training, you need to install the NVIDIA drivers, CUDA toolkit, and NVIDIA Container Toolkit. You can then configure persistence mode, power limits, and PCIe settings to ensure optimal performance and efficiency.

What is MIG partitioning and how does it work for multi-tenant A100 environments?▼

MIG partitioning slices high-end GPUs like A100 and H100 into isolated instances for multi-tenant environments. This allows multiple users to run separate AI workloads simultaneously on a single GPU, ensuring hardware-level isolation and optimal resource utilization.

Does this approach support GPU health monitoring and diagnostics via nvidia-smi?▼

Yes, GPU health monitoring and diagnostics integrate tools like nvidia-smi and DCGM. These tools track performance metrics, execute health checks, and verify multi-GPU topology to ensure your hardware operates correctly during demanding AI workloads.

Can I use the NVIDIA Container Toolkit for AI workloads without manual kernel tuning?▼

Using the NVIDIA Container Toolkit requires kernel tuning to achieve optimal performance for AI workloads. Addressing kernel parameters and verifying multi-GPU topology ensures your system handles LLM inference and training efficiently without hardware bottlenecks.

What's the best way to configure power limits and PCIe settings for multi-GPU topology?▼

The best way to configure power limits and PCIe settings is through automated performance tuning scripts that set persistence mode. Verifying multi-GPU topology ensures optimal communication between GPUs, maximizing efficiency for intensive LLM inference tasks.