We had a question regarding how the final scoring on the full dataset would be done. Specifically the email mentioned “When the competition ends, we will update the leaderboard using the entire test set, and your best Prescreened submission will be used to determine your place in the final rankings.” Does this mean that our model with best score in the prescreened arena will be used with the final data set or that all models submitted to the prescreened arena will be used and the best model will be chosen for our score?
Good question. It’s the second option you describe: all models submitted to the prescreened arena will be scored on the full test set and the best model will be used to determine rankings.