How to slice Kinetics400 training dataset? (pytorch)

Question

I am trying to run the official script for video classification. I want to tweak some functions and running through all examples would cost me too much time. I wonder how can I slice the training kinetics dataset based on that script. This is the code I added before train_sampler = RandomClipSampler(dataset.video_clips, args.clips_per_video) in the script: (let's say I just want to run 100 examples.)

tr_split_len = 100
dataset = torch.utils.data.random_split(dataset, [tr_split_len, len(dataset)-tr_split_len])[0]

Then when hitting train_sampler = RandomClipSampler(dataset.video_clips, args.clips_per_video) , it pops out the error:

AttributeError: 'Subset' object has no attribute 'video_clips'

Yeah, so the type of dataset converts from torchvision.datasets.kinetics.Kinetics400 to torch.utils.data.dataset.Subset. I understand. So how can I do it? (hopefully not the way using break in the dataloader loop). Thanks.

question from:https://stackoverflow.com/questions/65841394/how-to-slice-kinetics400-training-dataset-pytorch

Answer 1

It seems that torchvision.datasets.kinetics.Kinetics400 internally uses an object of class VideoClips to store the information about the clips. It is stored in the member variable Kinetics4000().video_clips.

The VideoClips class has a function called subset, that takes a list of indices and returns a new VideoClips object with only the clips with the specified indices. You could then just replace the old VideoClips object with the new one in your dataset.