Optimizes NVIDIA GPU AI inference and training with custom CUDA kernels for HuggingFace libraries.
npx skills add https://github.com/huggingface/kernels --skill cuda-kernels
Please help me install this Agent Skill. Skill: cuda-kernels Source: https://github.com/huggingface/kernels/tree/main/skills/cuda-kernels Command: npx skills add https://github.com/huggingface/kernels --skill cuda-kernels
This Skill provides optimized CUDA kernels and integration guidance to significantly accelerate AI model inference and training on NVIDIA GPUs, reducing latency and improving throughput.
diffusers and transformers libraries.Benchmark the performance of optimized kernels against the baseline implementation for video generation.
High-intent search queries and answers about installing and using this skill.
To optimize CUDA kernels for HuggingFace models, replace standard PyTorch operations with custom vectorized kernels targeting RMSNorm and GELU. This directly improves GPU compute efficiency and memory bandwidth utilization.
Yes, you can use custom CUDA kernels to accelerate Stable Diffusion video generation by up to 6%. The integration provides warp-level optimizations and precise memory hierarchy management for faster inference.
Yes, the GPU optimization approach supports NVIDIA H100, A100, and T4 architectures. It addresses numerical stability and memory bandwidth utilization challenges specific to these different GPU compute environments.
The best way to benchmark custom CUDA kernels against baseline PyTorch implementations is using the provided benchmarking scripts. These tools measure performance gains in latency and throughput for AI model inference and training.
Warp-level optimizations improve LLM inference latency by maximizing GPU compute efficiency and managing the memory hierarchy precisely. This reduces bottlenecks in standard operations during transformer model execution.
Yes, you need PyTorch, the accelerate library, and the HuggingFace transformers or diffusers packages. These dependencies provide the necessary framework interfaces to integrate the custom CUDA kernels into your workflows.
togethercomputerIntegrate an AOT CUDA/C++ kernel with Torch bindings and CMake.
danielfleischerBenchmark optimized CUDA kernels for diffusers and transformers on NVIDIA GPUs.
zechenzhangAGITrain Mixture-of-Experts models with DeepSpeed MoE and HuggingFace Transformers.
CHENyiru3Build and train custom neural networks with PyTorch for CPU and GPU environments.