Could you please specify the number of test cases for being able to predict inference time and be able to fit it to 8 hours(rule given by authors)?
Hi @ali.mammadov, the number of test cases is between 500 and 600 images.
Hi @ali.mammadov - I forgot to mention that you can also find the exact number of cases (541) using the
submission_format.csv file on the Data Download page.
Can you confirm that there is a public leaderboard (the one we see on the website) AND a private leader board (the one that will matter for the final ranking) ?
If so, can we know the number of cases in public and private set ?
Thank you very much.
Hi @Optimo, yes there is a public and private leaderboard. The public leaderboard is what you see during the competition and the private leaderboard is what you’ll see when the competition closes. We do not release the sample sizes for these.
Could you clarify how the final score is computed: do we have to pick our final submission or the final score is our best score in the private LB based on all our submissions ?
Thanks in advance
Hi @Optimo ,
The final score is based on your best score on the private leaderboard over all your submissions; you do not have to pick your final submission.
I was wondering, is there any way in which I can select a submission other than the one with highest leaderboard score?
Like, I think I got lucky with the top-scored one, but I don’t trust it too much, is there a way to ignore it and submit another one, even if it has a lower LB score?
@agaldran you do not need to pick your final submission as all your submissions are taken into account in the final LB.
Ah now I am reading it more slowly; so all submissions are scored and the one with highest score is automatically selected?
[That sounds like a bit of an advantage for people with lots of submissions, but ok]
So all submissions are scored and the one with highest score is automatically selected
Yep, your best score on the private test set is what will be used for the final leaderboard.
hat sounds like a bit of an advantage for people with lots of submission
Everyone has the same limit of 3 submissions per week.