k8s-launch-kit-troubleshoot

Diagnose NVIDIA Network Operator and Kubernetes networking failures from sosreport files.

14|5|Updated Nov 4, 2025
One-click install
npx skills add https://github.com/NVIDIA/k8s-launch-kit --skill k8s-launch-kit-troubleshoot
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: k8s-launch-kit-troubleshoot
Source: https://github.com/NVIDIA/k8s-launch-kit/tree/main/skills/k8s-launch-kit-troubleshoot
Command: npx skills add https://github.com/NVIDIA/k8s-launch-kit --skill k8s-launch-kit-troubleshoot

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps diagnose and resolve NVIDIA Network Operator and Kubernetes networking failures, reducing downtime caused by broken OFED drivers, missing virtual functions, failed pods, RDMA issues, and misconfigured network resources.

Core Features & Use Cases

  • Cluster Troubleshooting: Inspect Network Operator pods, NicClusterPolicy status, SR-IOV node states, device plugins, logs, events, and network resources.
  • Sosreport Analysis: Navigate diagnostic dumps covering cluster metadata, CRDs, operator components, node resources, networking state, and collection errors.
  • Failure Remediation: Identify likely causes and corrective actions for OFED crashes, VF creation failures, IP allocation problems, missing NetworkAttachmentDefinitions, stuck pods, and discovery failures.
  • Use Case: When a workload is stuck in ContainerCreating and cannot obtain a secondary network, use this Skill to inspect pod events, Multus logs, NetworkAttachmentDefinitions, CNI binaries, and SR-IOV resource availability.

Quick Start

Ask the k8s-launch-kit-troubleshoot skill to analyze the provided sosreport and identify the root cause of the Network Operator failure.

Frequently Asked Questions about k8s-launch-kit-troubleshoot

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I troubleshoot Kubernetes pods stuck in ContainerCreating with missing SR-IOV network resources?

To troubleshoot Kubernetes pods stuck in ContainerCreating, inspect pod events, Multus logs, NetworkAttachmentDefinitions, CNI binaries, and SR-IOV resource availability to identify why the secondary network cannot be obtained.

Why does the NVIDIA Network Operator fail to create virtual functions for RDMA workloads?

NVIDIA Network Operator virtual function creation failures for RDMA workloads are diagnosed by inspecting NicClusterPolicy status, SR-IOV node states, device plugins, and operator logs to pinpoint missing network resources or configuration errors.

How do I analyze a sosreport to diagnose OFED driver crashes in a Kubernetes cluster?

Analyze a sosreport for OFED driver crashes by navigating diagnostic dumps covering cluster metadata, CRDs, operator components, node resources, and networking state to identify the root cause of the failure.

Can I use this approach to fix IP allocation problems and missing NetworkAttachmentDefinitions?

Yes, you can fix IP allocation problems and missing NetworkAttachmentDefinitions by inspecting Kubernetes resources, operator logs, CRDs, node capabilities, and device-plugin state to determine corrective actions for network failures.

What is the best way to resolve NicClusterPolicy errors causing failed operator components?

Resolving NicClusterPolicy errors requires inspecting Network Operator pods, node capabilities, and structured diagnostic reports to identify likely causes and corrective actions for the failed operator components.