About the time limit

Video Similarity Challenge
1

Thanks again for the competition. I want to know whether the time of unzipping the submission (e.g. unzipping a 30G zip file will take some minutes) is included in

The submission must complete execution in less than 150 minutes (ten seconds per query for ~800 query videos plus overhead for the similarity search)

It seems not, but I want to ensure how the time is calculated in the docker. Thanks!

2

@wenhaowang Great question.

The limitation of 150 minutes includes 140 minutes for running inference (~10s / video for 832 query videos) and 10 minutes of overhead (similarity search, zip unpacking, etc.). This time limitation is applied equally for all participants regardless of submission size.

If you should find that unpacking your submission is substantially interfering with your submission evaluating in the provided window, we will consider increasing the time limitation for all participants; however, any increase will be subject to resource constraints and other considerations, so I don’t want to overpromise. We will discuss internally and come to a decision this week.

Thanks,
Chris

3

Thanks for that. Personally, I prefer the time not including other things (unzip, and others). I know they should subject to other restrictions. By the way, it seems the used zip is much slower than tar. My suggestion is only focusing on the feature extraction and similarity search. Please make a decision. Thanks

4

Hi, I want to confirm that the 140 minutes do not include the uploading time? Right? Thanks!

5

@wenhaowang I believe I now have answers to your questions:

  • The time to upload is not counted against your time limit
  • The time to unzip your submissions is included in the total runtime limit of the container, but due to the amount of overhead time we provide, unzipping submissions (even very large ones) still allows for inference to run up to the inference time limit.

We will add some logging output around the running of inference to provide additional documentation / feedback.

Let me know if you have any follow-up questions!

-Chris

6

There are no other questions. Thanks very much!