I sent my submission yesterday at 22:30, it’s 19:05 local time at the moment and there are still 5 submissions ahead of mine in the queue. So It will definitely take more that a full day to execute the submission. And I can guess that it’s becoming worse closer to the end of the competition.
So can we ask the organizers to add some scorers for this final period of competition before the queue completely stuck?
I suppose median wait time is more than 12 hours now. @rbgb
Do you consider to add another execution node?
23:30 - started to execute, 25 hours of waiting time + execution time
24 hours wait time here, still pending
Absolutely, we want you to be able to make the most of the last 2 weeks of the competition. We’re working on adding more capacity – stay tuned!
Ok, we doubled the capacity of the cluster from 2 to 4! You should start to see things moving.
It looks like in scaling up the cluster, 25 or so submissions that were in the queue at the time were erroneously marked from Pending to Started. These will not actually start processing until the ones ahead of them in the queue finish. So the wait time will be inaccurate, but those submissions should run and score without trouble. If you do see issues with those submissions, let me know in this thread. (You can resubmit, and hopefully not have to wait as long now that the cluster has more capacity.)
Thanks for scaling up the cluster! I suppose my submission was one of mentioned 25, it’s been 24 hours of waiting time and marked as starting for at least 6-7 hours. What do you think, should I cancel execution and resubmit or it has a chance to be executed soon?
From here it looks like those do get scheduled with their original order in the queue, so I think your best bet is to wait it out. Actually it looks like your recent submission may have just started?
You are right, terrible timing on my part, ha-ha . Thank you!