cuda-c-basics

To write and launch a CUDA-C kernel, define a __global__ function, allocate device memory, transfer data from host to device, execute the kernel across a grid and block configuration, and copy results back to the host.

The standard five-step CUDA kernel structure involves allocating GPU memory, copying input data from host to device, launching the kernel with specified grid and block dimensions, copying output data back to host, and freeing the allocated device memory.

CUDA memory management categorizes data by access scope and speed: global memory for host-device transfers, shared memory for inter-thread communication within a block, constant memory for read-only data, and registers for fast thread-local access.

To calculate global thread indices in a CUDA kernel, multiply the block index by the block dimension and add the thread index, ensuring each thread maps to a unique data element across the entire grid configuration.

No, PyTorch experience is not strictly required to learn CUDA-C kernel programming basics, though understanding tensor operations helps; this foundational skill focuses directly on writing C kernels and managing GPU memory hierarchies.

The best way to structure host-device data transfers is to use the standard five-step kernel pattern: explicitly allocate device memory, copy inputs to the GPU before launch, and retrieve outputs to the host immediately after kernel execution.