skills.rest
ViktorBarzin
ViktorBarzinCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

k8s-gpu-no-nvidia-devices

Diagnose and restore missing NVIDIA devices in Kubernetes GPU pods.

5|Updated Feb 7, 2021
Search Tags:#troubleshooting#kubernetes#gpu#cuda#nvidia#device-plugin
One-click install
npx skills add https://github.com/ViktorBarzin/infra --skill k8s-gpu-no-nvidia-devices
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: k8s-gpu-no-nvidia-devices
Source: https://github.com/ViktorBarzin/infra/tree/main/.claude/skills/k8s-gpu-no-nvidia-devices
Command: npx skills add https://github.com/ViktorBarzin/infra --skill k8s-gpu-no-nvidia-devices

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Kubernetes pods scheduled with GPUs sometimes see no NVIDIA devices inside the container, causing CUDA-enabled workloads to fall back to CPU despite GPU resource requests.

Core Features & Use Cases

  • Diagnose GPU injection issues in Kubernetes clusters and verify device visibility inside pods.
  • Validate NVIDIA device plugin status and perform safe restarts or resource reallocation to recover GPU access.
  • Use cases include resolving "CUDA not supported" or "no devices /dev/nvidia*" errors in GPU-enabled workloads.

Quick Start

Restore NVIDIA device visibility in a failing GPU pod by following the remediation steps.

Frequently Asked Questions about k8s-gpu-no-nvidia-devices

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
Why does my Kubernetes pod report CUDA not supported despite node GPU availability?▼

Kubernetes pods report CUDA not supported when the NVIDIA device plugin fails to mount devices. This Skill troubleshoots GPU pod visibility by diagnosing missing NVIDIA devices and verifying pod GPU resource requests and limits.

How do I fix missing /dev/nvidia* devices inside a Kubernetes container?▼

Fix missing /dev/nvidia* devices by validating the NVIDIA device plugin status and performing safe restarts. This Skill provides remediation steps to recover GPU access by checking device plugin health and reallocating resources.

What does the NVIDIA device plugin do for Kubernetes GPU workloads?▼

The NVIDIA device plugin exposes GPUs to Kubernetes containers, ensuring workloads access NVIDIA hardware. When the plugin fails, GPU-enabled workloads fall back to CPU despite having GPU resource requests configured on the node.

How do I check pod GPU requests and limits to restore device visibility?▼

Check pod GPU requests and limits to ensure Kubernetes schedules the workload correctly on GPU nodes. This Skill validates resource specifications alongside device plugin status to troubleshoot and restore missing NVIDIA device visibility.

When should I restart the NVIDIA device plugin to recover GPU access?▼

Restart the NVIDIA device plugin when pods show no NVIDIA devices despite node GPU availability. This Skill outlines safe restart procedures and resource reallocation checks to recover GPU access for CUDA-enabled workloads.

Related Skills

Explore More Software Engineering Skills→
racarracar
1

postgres-query

Execute SQL statements in PostgreSQL containers via automatic container detection.

Community
Advanced
RobThePCGuyRobThePCGuy
167

troubleshooting-assistant

Diagnose and resolve MCP, GPU, BigQuery, index, and import failures.

Community
Intermediate
BagelHoleBagelHole
46

gpu-kubernetes-operations

Manage GPU-backed Kubernetes clusters for AI inference and training.

Community
Advanced
KangOxfordKangOxford

multi-node-jax-ffi-debug

A structured guide for deploying and debugging multi-node JAX applications, emphasizing GPU cluster-specific considerations.

Community
Advanced