skills.rest
EvolvingLMMs-Lab
EvolvingLMMs-LabOFFICIAL
Explore all skills by this author →
𝕏@lmmslabGitHub

cu-lengths-attention-flow

Analyze cu_lengths attention boundaries across ViT and LLM stages in LLaVA-OneVision2.

1.2k|78|Updated Sep 16, 2025
Search Tags:#LLM#attention#cu_lengths#patch_positions#ViT#packed
One-click install
npx skills add https://github.com/EvolvingLMMs-Lab/LLaVA-OneVision-2 --skill cu-lengths-attention-flow
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cu-lengths-attention-flow
Source: https://github.com/EvolvingLMMs-Lab/LLaVA-OneVision-2/tree/main/.opencode/skills/cu-lengths-attention-flow
Command: npx skills add https://github.com/EvolvingLMMs-Lab/LLaVA-OneVision-2 --skill cu-lengths-attention-flow

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill provides a bilingual, in-depth guide to how cu_lengths controls attention boundaries across the ViT and LLM stages in LLaVA-OneVision2, clarifying why patch_positions grouping affects ViT but not LLM attention.

Core Features & Use Cases

  • Clarified attention semantics: explains non-packed full causal attention in the LLM and how packed cu_seqlens creates block-diagonal attention across sub-samples.
  • Practical debugging guidance: helps reason about offline packed vs. non-packed data, and how sample boundaries affect attention isolation.
  • Cross-modality reasoning: describes how ViT-level patch_positions interacts with LLM-level attention.

Quick Start

Inspect the forward-pass attention flow in LLaVA-OneVision2 and identify where cu_lengths and cu_seqlens determine attention domain.

Frequently Asked Questions about cu-lengths-attention-flow

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How does cu_lengths control attention boundaries in LLaVA-OneVision2?▼

In LLaVA-OneVision2, cu_lengths controls attention boundaries by generating cu_seqlens from the task_encoder, which determines block-diagonal attention isolation across sub-samples in the LLM and scopes attention domains in the ViT.

What is the difference between packed and non-packed attention in ViT and LLM stages?▼

Non-packed mode applies full causal attention in the LLM, while packed mode uses cu_seqlens to create block-diagonal attention across sub-samples. Sample boundaries and image tokens dictate how attention isolation arises in offline data analysis.

Why does patch_positions grouping affect ViT attention but not LLM attention?▼

patch_positions grouping scopes attention domains specifically in the ViT stage, while the LLM stage relies on cu_seqlens to establish block-diagonal attention isolation across sub-samples, bypassing patch grouping mechanisms.

How do I debug offline packed vs non-packed data in LLaVA-OneVision2?▼

Debug offline packed versus non-packed data by inspecting the forward-pass attention flow to identify where cu_lengths and cu_seqlens determine attention domains, clarifying sample boundary isolation effects across modalities.

What role does cu_seqlens play in Flash Attention for multimodal models?▼

cu_seqlens in Flash Attention defines block-diagonal attention boundaries across sub-samples in packed mode. Generated by the task_encoder via cu_lengths, it isolates attention domains and distinguishes sample boundaries from image tokens.

Related Skills

Explore More Software Engineering Skills→
sayalinvidiasayalinvidia

nemo-mbridge-perf-sequence-packing

Configure and validate packed sequences and long-context training in Megatron-Bridge.

Community
Advanced
EvolvingLMMs-LabEvolvingLMMs-Lab
1.2k

offline-packing-env-vars

Configure OFFLINE_PACKING_BMR and OFFLINE_PACKED_DATA for LLaVA-OneVision2 training.

Official
Advanced
tachyon-beeptachyon-beep
14

attention-mechanisms-catalog

Select attention mechanisms to reduce memory use for long sequences.

Community
Advanced
sjinkssjinks

vuln-triage-reachability

Trace WordPress plugin exploit paths from entry points to sinks.

Community
Advanced