Submission(s) for private LB

On Cloud N
1

Hi,

As it’s my first time here, I’m wondering if we need to select final submission(s) for private LB or if all submissions are scored in both public and private LB?
According to the official rules, I also understand that private LB is 100% with unseen data, correct?

Finally, are we allowed to download data used for public LB for pseudo labelling?

Thanks.

2 Likes
2

Good questions:

  1. No need to select a final submission.
  2. Correct, private leaderboard is 100% unseen data.
  3. No, we do not allow access to the public leaderboard data.
2 Likes
3

Thanks for clarifying! The private leaderboard is not just some fraction of the 10980 chips we’re testing on when we submit our models… is it?

This includes logging any and all metadata attached to the chips for further examination, correct? Especially in the case where the private leaderboard is some of the 10980 chips people can extract metadata from…

i.e. are there checks to make sure people are not writing out longitude/latitude to their log files, and submitting models that they know will do good on those biomes?

1 Like
4

Thanks for raising your concerns; as a practice, we don’t disclose all of our anti-cheating measures, but I will say that we monitor strictly for violating the rules of the competition both during the competition and when validating the prize-winning solutions. Competitors that violate the rules will be disqualified. You can rest assured that the best honest submissions will win!

2 Likes
5

Great, thanks for clarifying!

Just made me think the chips themselves were off limits, but made me worry about the rest of the info… Glad to know nothing at all from the 10980 chips but the public IoU can be used!