Dataloading at the moment is somewhat slow. Even with frame subsampling (Balint does 5x, Aiden does 20x), we find that:
- Unpacking video to frames takes a lot of space and memory
- The initial iteration through the dataset to get the mean image can take ~minutes.
- When feeding frames to the GPU, the limiting factor is how fast workers can pass frames to the GPU (GPU utilization is quite bursty).
Balint has also implemented a tarball based dataloader, which would take care of 1. We have also thought of using DALI to do GPU based data preprocessing, optionally directly from the video, which would take care of 2 and 3. DALI is apparently also compatible with the Webdataset format, which is what Balint's tarball based dataset uses.
Dataloading at the moment is somewhat slow. Even with frame subsampling (Balint does 5x, Aiden does 20x), we find that:
Balint has also implemented a tarball based dataloader, which would take care of 1. We have also thought of using DALI to do GPU based data preprocessing, optionally directly from the video, which would take care of 2 and 3. DALI is apparently also compatible with the Webdataset format, which is what Balint's tarball based dataset uses.