cuda-compat-vendor

Enable PyTorch operators on CUDA-compatible accelerators by bundling the vendor's libtorch_cuda.so.

12|18|Updated Apr 7, 2026
One-click install
npx skills add https://github.com/flagos-ai/Torch-FL --skill cuda-compat-vendor-flagos-ai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cuda-compat-vendor
Source: https://github.com/flagos-ai/Torch-FL/tree/main/.claude/skills/cuda-compat-vendor
Command: npx skills add https://github.com/flagos-ai/Torch-FL --skill cuda-compat-vendor-flagos-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Bringing up a new CUDA-compatible accelerator (MetaX, Hygon DCU, PPU) normally means writing hundreds of operator kernels. This Skill shows how to avoid that entirely by reusing the vendor's own CUDA-shaped torch build: extract its libtorch_cuda.so, preload it before importing torch, and box flagos tensors to the CUDA dispatch key with zero data copies. ## Core Features & Use Cases - Compatibility verification: Mechanically prove a vendor ships a usable libtorch_cuda.so by inspecting ATen symbols and dumping the dispatcher table before committing to this path. - Vendor .so extraction and wheel bundling: Download (never pip-install) the vendor torch wheel, stage libtorch_cuda.so and libc10_cuda.so into .libtorch_cuda_assets, and let setup.py bundle them into the built wheel. - Load-timing and build wiring: Enforce the mandatory LD_PRELOAD-before-import-torch constraint via scripts/vendor/with_cuda_libtorch.sh, and wire the correct FLAGOS_ACCELERATOR branch (DCU-style glob, PPU-style detection, or MetaX-style shim) with a g++-only, no-nvcc build. - Use Case: You receive a Hygon DCU machine whose DTK ships a CUDA-compatible torch. Follow the steps to confirm dispatcher entries for mm/add/_softmax/bmm, bundle the extracted libraries, and run the operator integration suite with tensors staying on the flagos device. ## Quick Start Verify the vendor's libtorch_cuda.so registers CUDA kernels for aten::mm, then extract it from the vendor wheel and run the torch_fl operator tests through the with_cuda_libtorch.sh preload wrapper.

Frequently Asked Questions about cuda-compat-vendor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I enable PyTorch operators on a CUDA-compatible accelerator without writing kernels?▼

Extract libtorch_cuda.so and libc10_cuda.so from the vendor's own torch wheel, LD_PRELOAD them before importing torch, and box flagos tensors to the CUDA dispatch key. The vendor's compiled kernels run through the public at:: API with no data copies.

How to verify a vendor GPU is CUDA-compatible before committing?▼

Check that the vendor ships a libtorch_cuda.so with ATen symbols using nm, then load it with ctypes and run torch._C._dispatch_dump("aten::mm"). If CUDA entries appear for mm, add, _softmax, and bmm, the boxing path is viable.

Does the vendor libtorch_cuda.so version need to match installed torch?▼

Yes, the match must be exact: torch 2.9.0+cpu pairs only with a 2.9.0 CUDA build. A minor mismatch does not fail at load time but causes an ABI crash later, which is far harder to debug.

Why must libtorch_cuda.so be loaded before import torch?▼

PyTorch builds its dispatcher table at import time, so loading the library afterwards leaves no CUDA entries registered and no later CDLL call repairs it. The failure is silent: ops quietly run on CPU or raise device-mismatch errors far from the cause.

What causes 'Allocator not initialized for device' on the first CUDA op?▼

This path never calls torch.cuda._lazy_init(), so PyTorch's CUDA caching allocator is not primed; out-variant ops like mm.out run first can trigger it. Accept or xfail those tests, or prime once with a throwaway functional op at import.

When should I use native kernel development instead of CUDA compatibility boxing?▼

Use the native operator backend when the vendor does not ship a libtorch_cuda.so whose ATen symbols register under the CUDA dispatch key. The dispatcher dump test in Step 1 is decisive; if no CUDA entries appear, boxing is not viable.