Verified:GitHub

multi-node-jax-ffi-debug

Name: multi-node-jax-ffi-debug
Availability: InStock
Author: kangoxford

A structured guide for deploying and debugging multi-node JAX applications, emphasizing GPU cluster-specific considerations.

1|Updated Apr 23, 2026

Search Tags:#debugging #jax #cuda #ffi #slurm #jax-distributed

Ready to run

npx skills add https://github.com/KangOxford/auto-quant-research --skill multi-node-jax-ffi-debug

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: multi-node-jax-ffi-debug
Source: https://github.com/KangOxford/auto-quant-research/tree/main/execution-layer/skills/multi-node-jax-ffi-debug
Command: npx skills add https://github.com/KangOxford/auto-quant-research --skill multi-node-jax-ffi-debug

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Systematically diagnose and resolve multi-node JAX + CUDA FFI kernel failures when using shard_map on SLURM clusters.

Core Features & Use Cases

Provides a structured six-step fix sequence to diagnose and repair common multi-node JAX FFI issues.
Guides reliable environment propagation, explicit GPU allocation, distributed initialization, and all-rank synchronization to prevent deadlocks.
Includes practical coordinator resolution strategies using scontrol and robust wrapper scripts to ensure correct per-node setups.

Quick Start

Follow the six-step checklist to stabilize a multi-node JAX FFI run on SLURM by wrapping environment setup, requesting GPUs, enabling all ranks in distributed init, ensuring collective ops, and resolving the coordinator with scontrol.

Frequently Asked Questions about multi-node-jax-ffi-debug

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I debug multi-node JAX FFI kernel failures on a SLURM cluster?▼

To debug multi-node JAX FFI kernel failures on a SLURM cluster, follow a structured six-step sequence covering environment propagation, explicit GPU allocation, distributed initialization, and all-rank synchronization to prevent deadlocks.

Why does my JAX distributed initialization deadlock during collective operations?▼

JAX distributed initialization deadlocks during collective operations when not all ranks participate in the collectives or when coordinator discovery fails. Ensuring all ranks are enabled in distributed initialization and verifying coordinator resolution with scontrol prevents these synchronization failures.

How do I port a single-GPU JAX FFI kernel to a multi-node deployment?▼

Porting a single-GPU JAX FFI kernel to multi-node deployment requires addressing device visibility, local_device_count, and rank participation in collectives. Use wrapper scripts for environment setup and ensure reliable per-node configurations to stabilize the distributed run.

What is the best way to resolve the coordinator node for JAX distributed runs on SLURM?▼

The best way to resolve the coordinator node for JAX distributed runs on SLURM is using scontrol commands within robust wrapper scripts. This ensures correct per-node setups and reliable coordinator resolution, preventing initialization failures across the multi-node environment.

Does shard_map work with multi-node JAX CUDA FFI kernels on SLURM?▼

shard_map works with multi-node JAX CUDA FFI kernels on SLURM when you explicitly request GPUs, propagate the environment correctly, and ensure all ranks participate in collective synchronization. Properly configuring local_device_count and device visibility is required for stable execution.

Why is local_device_count returning zero for my multi-node JAX FFI run?▼

local_device_count returns zero in multi-node JAX FFI runs due to incorrect device visibility or missing explicit GPU allocation in the SLURM wrapper script. Requesting GPUs explicitly and ensuring proper environment propagation remediate this device visibility issue.