Video DataLoader Pipeline (Decord + tf.data)

What problem does it solve?

This skill provides a complete blueprint for building production-grade video data pipelines, unifying Decord-based PyTorch loading with tf.data-based TPU pipelines to streamline multi-backend training workflows.

Core Features & Use Cases

• Multi-backend support: PyTorch with Decord VideoReader and tf.data with sharded TFRecords, enabling GPU and TPU training paths.
• Deterministic sampling and augmentation: Robust temporal frame sampling with fixed stride and random starts, plus GPU-accelerated augmentations and deterministic eval transforms.
• Lightning DataModule integration: Clean interfaces for train/val/test datasets and dataloaders with correct distributed sampling behavior.
• Templates and references: Ready-to-use templates for VideoDataset, TFRecordConverter, tf.data pipelines, and reference materials to accelerate integration.

Quick Start

Create a video dataloader pipeline using Decord and tf.data, then wire it into a Lightning DataModule.