I realize there is probably a limit of 1 submission per 5 days due to compute resource constraint concerns, but it would be super helpful to be allowed to submit more frequently, especially at the beginning of the competition as we get used to the new environment.
I noticed they do let you re-submit immediately if your code fails to make predictions due to an error. So that at least lets you get inference working.
If the limit isn’t increased, the next best thing is to replicate their scoring and keep one season for evaluation. I get a pretty good estimate scoring with np.mean([sklearn.metrics.log_loss(test[c], preds[c]) for c in classes]) - seemed to match up with the leaderboard score.