According to the Announcement there has been some change on the data. I’m wondering if it’s possible to tell which files were changed. Changing everything in the pipeline is very time consuming. It would be really helpful if the host could provide us the change-log of the updated files. Then we we’ll be able to change only the updated files instead of all the files.
Also I’m wondering if test data has been updated or not …
@awsaf49 The only files changed were the downsampled videos provided to competitors, per the announcement. This includes the downsampled videos for both the train set videos and the test set videos. This means the table video_access_metadata.csv also changed because it includes checksum hashes for all of the downsampled videos.
Otherwise, everything else has remained the same. If your pipeline uses downsampled videos you should re-download the downsampled videos. If your pipeline uses only the full length videos, no changes are needed.
@kwetstone Actually I tried re-downloading only the test data and resubmitting using that data and previous model but I didn’t notice that much of a change on the leader-board score. So, all the videos has been changed or some specific videos?
We updated all of the downsampled videos, but I would not anticipate a large change in performance. Only some of the downsampled videos had incorrect timing, and those that did were only off by 1-2 seconds (per the announcement). Hope that helps!
