Is that the last one?
The one with the highest value on private leaderboard among all submissions?
The one with the highest value on public leaderboard among all submissions?
For e.g. I want to submit model which has the highest value on my private validation. Do I need to make it last submission? In general do we need to make last submissions to fix the results we think are the best ones?
@kuddai The submission with the highest value on the private leaderboard among all submissions will be used for final ranking. See the detailed description about winner selection in the rules.
Oh, so it is actually beneficial to make more submissions.
Usually it is not worth it as you would overfit for public leadboard. But here I guess it is one of the best strategies there is.
the objective of hidden private leaderboard is to prevent overfitting of private test data.
For drivendata, “submission with the highest value on the private leaderboard among all submissions will be used for final ranking.”.
Actually, this would overfit private data. e.g. one can submit a model with threshold sweeping from a range of threshold values like 0.1, 0.2, 0.3, 0.4, 0.5,… Here the overfitted threshold will be selected as “winning submission” in private
you are right, I overlooked “private” part. But it is even worse than, as you can’t select final submissions. For e.g. if you wanted to choose model which showed the highest result on the cross validation.