Diagnose and fix incorrect gradients in differentiable NVIDIA Warp programs.
npx skills add https://github.com/NVIDIA/skills --skill warp-debug-gradients
Please help me install this Agent Skill. Skill: warp-debug-gradients Source: https://github.com/NVIDIA/skills/tree/main/skills/warp-debug-gradients Command: npx skills add https://github.com/NVIDIA/skills --skill warp-debug-gradients
Differentiable simulations and training loops built on NVIDIA Warp often fail silently: the forward pass looks healthy while wp.Tape gradients are zero, NaN, doubled, or subtly wrong due to taping-pattern bugs like buffer reuse, missing requires_grad, or faulty custom adjoints. This Skill provides an evidence-driven workflow to verify gradients against finite differences, localize the root cause, and apply a minimal verified fix.
Ask your agent to debug why training through your Warp kernels diverges or produces wrong gradients, pointing it at your reproduction script.
High-intent search queries and answers about installing and using this skill.
Enable wp.config.verify_autograd_array_access under an active tape to catch write-after-read overwrites, then run an end-to-end wp.autograd.gradcheck comparing autodiff against finite differences at the true optimization inputs. Match the error signature (zero, NaN, exact factor, subtle drift) to known bug patterns before fixing.
Common causes include missing requires_grad=True on an array in the chain (wp.zeros defaults to False), enable_backward=False, a loss not connected to the tape, or reading grads after tape.zero(). Piecewise-constant ops like wp.round also correctly produce zero gradients, requiring a straight-through estimator instead.
Yes. Diagnosis runs the user's reproduction on any functioning device, CPU or CUDA, and the bug classes involved (overwrites, missing requires_grad, double accumulation) are taping properties independent of device. Only float atomic accumulation ordering differs between CPU and CUDA.
Warp >= 1.13 is the minimum, with >= 1.17 recommended for reliable verification. Copy-adjoint accumulation, overwrite-warning call sites, read-flag lifetime, and gradcheck's restore_inputs behavior changed in 1.17, so older versions need manual harness workarounds.
Do not use it for forward-only Warp work such as kernel authoring, rendering, or performance tuning, Warp build or installation problems, or autograd issues in other frameworks like pure PyTorch with no Warp involvement.
wp.autograd.gradcheck_tape validates each recorded launch in isolation, so it is structurally blind to inter-kernel overwrite bugs and silently skips kernels compiled with enable_backward=False. A clean per-kernel pass with wrong end-to-end gradients points at the taping pattern between kernels.
tensormuxClassify warp divergence in CUDA kernels and estimate cost with Nsight Compute metrics.
plurigridCompare empirical weight distributions to Gibbs distributions for Langevin training convergence.
mindspore-aiOptimize Triton CUDA kernels with performance strategies and API usage rules.
TorchedHatTrace PyTorch AOT Autograd FX graphs to debug compiled training workflows.
