skills.rest
dawsonblock
dawsonblockCOMMUNITY
Explore all skills by this author →
GitHub

optimizing-attention-flash

Automate transformer attention optimization with Flash Attention on PyTorch.

Updated Apr 26, 2026
Search Tags:#pytorch#transformers#flash-attention#gpu-memory#h100#attention-optimization
One-click install
npx skills add https://github.com/dawsonblock/HERMY --skill optimizing-attention-flash-dawsonblock
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: optimizing-attention-flash
Source: https://github.com/dawsonblock/HERMY/tree/main/hermes-agent-2026.4.23/optional-skills/mlops/flash-attention
Command: npx skills add https://github.com/dawsonblock/HERMY --skill optimizing-attention-flash-dawsonblock

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Optimizes transformer attention to deliver faster inference/training and lower memory usage, enabling longer contexts and more efficient workloads.

Core Features & Use Cases

  • Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.
  • Delivers 2-4x speedup and 10-20x memory reduction for long-sequence transformers.
  • Use cases include training with long sequences, memory-constrained inference, and rapid experimentation with attention variants.

Quick Start

Install flash-attn and enable the provided backend to accelerate attention in your transformer models.

Frequently Asked Questions about optimizing-attention-flash

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize transformer attention for faster training and lower memory usage?▼

Optimize transformer attention by applying Flash Attention to accelerate PyTorch training and inference, delivering 2-4x speedup and 10-20x memory reduction for long-sequence models.

What is the best way to handle long-context transformer models without running out of GPU memory?▼

Handling long-context models requires Flash Attention with sliding-window and multi-query attention support on PyTorch, reducing memory usage by 10-20x while maintaining rapid experimentation with attention variants.

Do I need an H100 GPU to use Flash Attention 2 and 3 with PyTorch?▼

Using Flash Attention with PyTorch requires CUDA-capable hardware, but FP8 support specifically requires an H100 GPU. Standard Flash Attention 2/3 and SDPA can run on other CUDA-capable hardware.

Does PyTorch native SDPA work with sliding window attention for memory-constrained inference?▼

PyTorch native SDPA supports sliding window attention for memory-constrained inference, allowing rapid experimentation with attention variants to improve speed and reduce memory overhead.

Can I use FP8 attention optimization for training long sequences?▼

FP8 attention optimization supports training with long sequences on H100 hardware, delivering 2-4x speedup and 10-20x memory reduction while applying Flash Attention 2/3 techniques.

Related Skills

Explore More Software Engineering Skills→
qiphonqiphon
2

V3 Performance Optimization

Optimize v3 performance targets with Flash Attention, HNSW indexing, and benchmark comparisons.

Community
Advanced
ihatesea69ihatesea69
3

optimizing-attention-flash

Optimize transformer attention with Flash Attention for PyTorch models.

Community
Advanced
zhuangbiaoweizhuangbiaowei
2

optimizing-attention-flash

Optimize transformer attention with Flash Attention for speed and memory efficiency.

Community
Advanced
JamesFincherJamesFincher

optimizing-attention-flash

Optimize transformer attention with Flash Attention to reduce GPU memory usage.

Community
Advanced