vae-opt

Validates and implements bit-exact multi-card sharding for VAE/TAE video decoders.

14|5|Updated Mar 30, 2026
One-click install
npx skills add https://github.com/Ascend/MindIE-SD --skill vae-opt-ascend
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: vae-opt
Source: https://github.com/Ascend/MindIE-SD/tree/main/.agents/skills/vae-opt
Command: npx skills add https://github.com/Ascend/MindIE-SD --skill vae-opt-ascend

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, numpy, av, and includes scripts (resource) and references (resource) components.

What problem does it solve? When running video generation across multiple GPUs/NPUs, every card often redundantly decodes the same VAE/TAE output, and naive attempts to shard the decoder silently corrupt frames or deadlock. This Skill determines whether a decoder can be sharded along a given axis, how to shard it bit-exactly, and whether the sharding is worth the communication cost. ## Core Features & Use Cases - Independence Auditing: Audits every operator for cross-axis coupling (e.g., recursive MemBlock prefix dependencies) to decide between frame-axis sharding, spatial tiling with halo, or state-carrying sliced decoding. - Four-Gate Verification: Enforces CPU bitwise equivalence (max|d| = 0), end-to-end md5 comparison, frame-level health metrics (high-frequency energy, frame diffs, frozen segments), and exchange-budget accounting before claiming losslessness. - Silent-Failure Traps: Documents pitfalls like all_gather_into_tensor writing to stack copies, repeat_interleave over-allocating memory, rank-dependent gating deadlocks, and CANN nearest-neighbor upsample defects on large batches. - Use Case: You run 8-card video generation where each card decodes the full video. Use this Skill to prove the decoder is frame-independent, shard it across cards, verify 12/12 md5 matches, and cut decode time to roughly a quarter. ## Quick Start Ask the AI to check whether your VAE decoder can be sharded across cards without changing the output, and to run the CPU bitwise equivalence check before deploying.

Frequently Asked Questions about vae-opt

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I shard VAE decoding across multiple GPUs without changing the output?

First audit every operator for cross-axis coupling: if no operator crosses the frame axis, shard along frames with no halo. If the decoder has recursive memory blocks, use state-carrying sliced decoding that passes each block's last-frame input activation across segment boundaries, then verify bitwise equivalence on CPU.

How do I prove sharded decoding is bit-exact versus single-card output?

Run four verification gates: CPU whole-vs-sliced comparison requiring max|d| = 0, end-to-end md5 matching across requests, frame-level health metrics (high-frequency energy, frame diffs, frozen segments), and exchange-budget accounting. Note that md5 match alone only proves the pipeline output is unchanged, not decoder fidelity.

Should I shard video decoding along the frame axis or spatial axes?

The choice is dictated by which operators couple which axes. Frame-axis sharding is preferred when nothing crosses time since it needs no halo and minimal communication; spatial tiling with halo sized to the receptive field works when only spatial convolutions couple pixels; recursive networks need state-carrying sequential slicing instead.

Why does my sharded VAE decoding show no speedup or wrong results?

First prove sharding actually happened by printing state values (pieces count, per-shard shapes, whether state is None), since gating misreads cause silent fallback to full redundant decoding. Then run the CPU bitwise check; nonzero max|d| usually means state stored block output instead of input, missing clone, or cut points off latent-frame boundaries.

When is multi-card VAE decode sharding not worth doing?

Skip it when decoding is under about 5 percent of end-to-end time, since DiT denoising and host overhead dominate. Also avoid it when the exchange budget fails: gathering fp32 video tensors costs gigabytes per card, so prefer uint8 exchange or writing tiles directly to the encoder.

Can chunked decoding code like chunk_latents prove a decoder is shardable?

No. Stock implementations with chunk_latents and trim values perform boundary artifact cropping, not independent computation. Shardability must be proven per operator by checking whether each output element depends on inputs outside the slice boundary.