I realize there is probably a limit of 1 submission per 5 days due to compute resource constraint concerns, but it would be super helpful to be allowed to submit more frequently, especially at the beginning of the competition as we get used to the new environment.
Are there any plans to increase it?
I noticed they do let you re-submit immediately if your code fails to make predictions due to an error. So that at least lets you get inference working.
If the limit isn’t increased, the next best thing is to replicate their scoring and keep one season for evaluation. I get a pretty good estimate scoring with np.mean([sklearn.metrics.log_loss(test[c], preds[c]) for c in classes]) - seemed to match up with the leaderboard score.
All of my submissions failed, and then I am not able to submit anymore. It should be one successful submission right?
That was certainly the case for me - I had two failures where my zip file didn’t have the correct structure - then 1 success.
@jigsaw @brismith It sounds like you may have encountered the same issue as in a different thread. Here’s that thread and the fix:
If you have steps you took where you saw the issue, let us know.
Can we increase the submission limit to at least 1 in 3 days? 1 in 5 days would leave us with very less submissions.
Thanks @bull, but my issues were entirely user error 
my zip file didn’t have the right structure.
Yeah I’m late to this and now have little chance of submitting more than a couple jobs…
We have just increased the submission limit to one submission every day for the rest of the competition. Enjoy!
Missed this! Ah well Very kind of you to have given folks a chance to get some final tweaking done. Will we be able to make submissions after the competition closes?