Configure FlashAttention variants and alternative backends for CUDA GPUs.
npx skills add https://github.com/tylertitsworth/skills --skill flash-attention
Please help me install this Agent Skill. Skill: flash-attention Source: https://github.com/tylertitsworth/skills/tree/main/flash-attention Command: npx skills add https://github.com/tylertitsworth/skills --skill flash-attention
Enables selecting and configuring high-performance attention backends (FlashAttention 2/3, SDPA, PagedAttention, Ring Attention) for ML workloads on modern GPUs, reducing memory footprint and increasing throughput.
To begin, identify your GPU architecture and desired throughput and configure the appropriate attention backend in your training script or inference pipeline.
High-intent search queries and answers about installing and using this skill.
PyTorch attention backend selection depends on your GPU architecture, dtype, and workload. FlashAttention 2 or 3 maximizes training throughput, while SDPA provides reliable fallbacks for unsupported hardware configurations or specific head dimension limits.
FlashAttention optimizes training and inference speed by reducing memory footprint. SDPA serves as a framework default fallback, whereas PagedAttention specifically optimizes KV-cache memory management for long sequence inference workloads on compatible CUDA-enabled GPUs.
FlashAttention 3 does not work universally. Backend selection enforces strict compatibility checks validating specific GPU architectures, supported data types, and head dimension limits before applying the attention mechanism to your workload.
To configure Ring Attention in Hugging Face transformers, apply runtime integration tips to select the appropriate attention backend. This optimizes memory efficiency for long sequences by leveraging framework defaults and optional features like window_size and alibi.
Use SDPA fallback instead of FlashAttention when hardware compatibility checks fail due to GPU architecture, dtype, or head_dim limits. SDPA provides the framework default attention implementation when high-throughput FA2 or FA3 backends are unsupported.
anilcan-karaOptimize transformer attention with Flash Attention for PyTorch and CUDA GPUs.
superfhpOptimize transformer attention with Flash Attention for long sequences.
Supporter09Accelerate transformer attention and reduce GPU memory usage in PyTorch and Hugging Face Transformers.
MesferAliOptimize transformer attention with Flash Attention for PyTorch models.
