Will there be a hidden private test set?

Youth Mental Health: Automated Abstraction
1

Will there be a hidden private test set or the final ranking will be determined by the open leaderboard?

Thanks!

2

Hi @wdong , there is a portion of the test set that is hidden, and the public and hidden portions of the test set combined determine the winners. (The current leaderboard does not reflect performance on the hidden portion, but after the competition closes the leaderboard will reflect performance on both portions). Good luck!

3

Thanks for replying. Are we to select one of our submissions for final ranking or it’s automatically picked for us?

4

Your best-performing submission will be selected automatically.

1 Like
5

Wait, the best submission for the private leaderboard is automatically selected? This means submitting more simply increases the chance to win. Please require users to select a fixed number of submissions to avoid encouraging unnecessary seed-changing submissions.

6

Participants are limited to a maximum of three submissions a day. Note that the limit does not apply to smoke test submissions.