skills.rest
handsomelong922
handsomelong922COMMUNITY
Explore all skills by this author →
GitHub

optimizing-attention-flash

Automate transformer attention optimization with Flash Attention for long sequences on GPUs.

Updated Apr 3, 2026
Search Tags:#pytorch#transformers#gpu-acceleration#memory-optimization#fp8#flash-attention#h100
One-click install
npx skills add https://github.com/handsomelong922/my-codex-skills --skill optimizing-attention-flash-handsomelong922
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: optimizing-attention-flash
Source: https://github.com/handsomelong922/my-codex-skills/tree/main/skills/flash-attention
Command: npx skills add https://github.com/handsomelong922/my-codex-skills --skill optimizing-attention-flash-handsomelong922

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires flash-attn, torch, transformers, and includes references (resource) components.

What problem does it solve?

Optimizes transformer attention by using Flash Attention to dramatically reduce memory usage and increase throughput during training and inference on long-context sequences.

Core Features & Use Cases

  • Improves speed by 2-4x and reduces memory by 10-20x for transformer attention on long sequences (>512 tokens).
  • Supports PyTorch native SDPA, the flash-attn library, H100 FP8, and sliding window attention for local attention scenarios.
  • Use cases include training and deployment of large language models with long documents, streaming inference, and memory-constrained environments.

Quick Start

Enable Flash Attention in your transformer model to accelerate attention computation and reduce memory usage.

Frequently Asked Questions about optimizing-attention-flash

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize transformer attention for long context sequences in PyTorch?▼

Optimize transformer attention in PyTorch by enabling Flash Attention to reduce memory usage and increase throughput during training and inference on sequences exceeding 512 tokens.

What's the best way to reduce GPU memory bottlenecks for large language models?▼

Reduce GPU memory bottlenecks for large language models by applying Flash Attention, which delivers 10-20x memory savings and 2-4x speed improvements for transformer attention computations.

Does Flash Attention work with H100 FP8 and sliding window attention?▼

Flash Attention works with H100 FP8 and sliding window attention, supporting PyTorch native SDPA, the flash-attn library, and local attention scenarios for streaming inference.

Can I use PyTorch SDPA instead of the flash-attn library for attention optimization?▼

You can use PyTorch native SDPA as an alternative to the flash-attn library for attention optimization, as both are supported for accelerating attention computation and reducing memory usage on CUDA-enabled GPUs.

When do I need Flash Attention for transformer training and inference?▼

You need Flash Attention for transformer training and inference when processing long context sequences over 512 tokens where memory becomes a bottleneck, achieving dramatic memory reductions and throughput increases.

What are the dependencies required to enable Flash Attention in transformers?▼

Dependencies required to enable Flash Attention in transformers include PyTorch, the flash-attn library or PyTorch SDPA, and optional FP8 support, all operating on CUDA-enabled GPUs.

Related Skills

Explore More Software Engineering Skills→
tachyon-beeptachyon-beep
14

attention-mechanisms-catalog

Select attention mechanisms to reduce memory use for long sequences.

Community
Advanced
brittaniebuffiecsubrittaniebuffiecsu
1

optimizing-attention-flash

Optimize PyTorch transformer attention with Flash Attention for long sequences.

Community
Advanced
eyadsibaieyadsibai
7

llm-training

Compare frameworks and techniques for training and finetuning large language models.

Community
Intermediate
TeoSlayerTeoSlayer
7

pilot-directory

Build and maintain a local directory of agents and their metadata.

Community
Intermediate