Number of cases

VisioMel Challenge
1

Could you please specify the number of test cases for being able to predict inference time and be able to fit it to 8 hours(rule given by authors)?

1 Like
2

Hi @ali.mammadov, the number of test cases is between 500 and 600 images.

Good luck!

3

Hi @ali.mammadov - I forgot to mention that you can also find the exact number of cases (541) using the submission_format.csv file on the Data Download page.

Good luck!

4

Hello,

Can you confirm that there is a public leaderboard (the one we see on the website) AND a private leader board (the one that will matter for the final ranking) ?

If so, can we know the number of cases in public and private set ?

Thank you very much.

5

Hi @Optimo, yes there is a public and private leaderboard. The public leaderboard is what you see during the competition and the private leaderboard is what you’ll see when the competition closes. We do not release the sample sizes for these.

6

Hi,

Could you clarify how the final score is computed: do we have to pick our final submission or the final score is our best score in the private LB based on all our submissions ?

Thanks in advance

7

Hi @Optimo ,

The final score is based on your best score on the private leaderboard over all your submissions; you do not have to pick your final submission.

1 Like
8

Hello!

I was wondering, is there any way in which I can select a submission other than the one with highest leaderboard score?

Like, I think I got lucky with the top-scored one, but I don’t trust it too much, is there a way to ignore it and submit another one, even if it has a lower LB score?

Cheers,

9

@agaldran you do not need to pick your final submission as all your submissions are taken into account in the final LB.

1 Like
10

Ah now I am reading it more slowly; so all submissions are scored and the one with highest score is automatically selected?

[That sounds like a bit of an advantage for people with lots of submissions, but ok]

1 Like
11

So all submissions are scored and the one with highest score is automatically selected

Yep, your best score on the private test set is what will be used for the final leaderboard.

hat sounds like a bit of an advantage for people with lots of submission

Everyone has the same limit of 3 submissions per week.