Video DataLoader Pipeline (Decord + tf.data)

Build a video dataloader pipeline using Decord for PyTorch and sharded TFRecords for tf.data to enable multi-backend GPU and TPU training workflows. The pipeline unifies both paths with consistent video metadata caching and robust frame sampling.

Temporal frame sampling in a video dataloader is handled using robust sampling with fixed stride and random starts. This approach provides deterministic eval transforms and supports GPU-accelerated augmentations during training.

Yes, the video pipeline integrates directly with PyTorch Lightning DataModules. It provides clean interfaces for train, val, and test datasets with correct distributed sampling behavior for multi-GPU training.

Yes, the tf.data video pipeline requires converting videos to sharded TFRecords. The solution includes ready-to-use TFRecord conversion templates and configurable video dataset definitions to streamline this process.

Convert a video dataset to TFRecords using the provided TFRecordConverter templates. These templates support sharded output creation and include validation hooks with documented defaults to ensure data integrity for TPU training.

Video dataloader pipelines need metadata caching to avoid repeatedly scanning large video files for frame counts and timestamps. Caching this metadata accelerates dataset initialization and ensures consistent frame sampling across training epochs.