Is the leaderboard test set the final test set?

Countable Care
1

It seems like that is the case. Please clarify. Thank you.

2

Hi Cristoph,

Good question. As the competition rules mention, the final scoring is based on a “private leaderboard.” When the competition ends, this score becomes public (see Box-plots for Education as an example).

The public leaderboard for this competition–i.e., what you see on the leaderboard now–is based on a portion of the test set. The final score–i.e., the private leaderboard–is based on the entire test set.

TLDR: no, the leaderboard is not the final score.

Peter

3

Hello! How big is public leaderboard’s test set?

4

Hi @1aguschin generally we use between 1/3 to 1/2 of the test set data in the public leaderboard.

5

Thanks for your reply. One more question: Is the final ranking of the private leaderboard based on the best submission or on the last submission?

6

Only your best submission is counted towards ranking on the private leaderboard.