Hey,
Should we finalize and lock the models before the 17th or we can update the models after the 1st part?
after the 17th is there any restriction on, modeling including validation set?
Hi @Kugan,
Your models do not need to be locked at the end of Phase 1, and training on the validation set is allowed.
I just got to know about the competition, so is it mandatory to submit a model by 17th of march?
Hi @ethanhunt923, welcome! It is not mandatory to make a submission before March 17th. You can join the competition and make a first submission any time before the competition closes on April 18, 2022 at 11:59 p.m. UTC
1 Like
May I ask, how the submission for the final scoring is selected? Is it the last submission, or we can choose one or multiple submissions to be used for the final scoring?
Hi @dmytro. For this challenge, your best-performing submission on the test set will automatically be selected. No additional action from you is required after submitting predictions.
I know it’s very late in the competition, but I’d like to suggest removing the restriction for the number of submissions, but using for scoring only the last N (2 is common, but maybe 6 to approximately match the current situation).
I believe this would be beneficial for both the competition host (if someone had an idea after using all 6 submissions available during the last two weeks, he/she would still be able to submit it) and for competitors (the unfair advantage of earlier extra submissions with the relatively small test sam dataset would be removed, better control of what to be used for submission).
The validation dataset is published, so the leaderboard score from extra submissions does not give you any new information (can be used only for testing the submission format etc).
Hi @dmytro, we won’t be making any changes to this competition’s rules at this time. Thank you for your understanding. Thank you also for your feedback towards future competitions.
1 Like