Hi,
Can we expect any updates on the Final Prize Stage soon? Will the deadline stay the same, given that the rules for the final stage are still not clarified? Unfortunately, I don’t have a strong motivation to work on my solution further when the exact rules are still unknown.
1 Like
Hi @progin,
Thanks for your patience. We’re finalizing some details for the Final Prize Stage and the website should be available soon. The deadline will be moved, and currently that is planned to be moved to March 28.
2 Likes
Hi @jayqi ,
Do you have any updates about the evaluation of the hindcast stage?
Hi @RomanChernenko,
We have no updates about winners to share yet. Our judging panel is currently reviewing the model reports for the Hindcast Stage. After finalists are selected, they will be notified privately, and we will go through a process to collect their information and their model training code, and to verify their solutions as rules-compliant and reproducible. We won’t be ready to publicly announce Hindcast Stage winners until after that process is complete.
1 Like
Given that the performance of the top teams is quite high, our chance to win a prize is quite low.
@jayqi Is it possible that each team in the forecast LB could have a small gift (let’s say $ 500) to compensate the writing efforts in case the team does not win any prizes ?
1 Like
Hi @motoki,
Currently, there are no plans to award smaller prizes for participation.
However, please see the additional information regarding the prizes in the Final Prize Stage that was recently released. In particular:
- For the main Overall Prizes (5 prizes totaling $275,000), the leaderboard performance is only 40% of the evaluation criteria. Other criteria including Rigor, Innovation, and Generalizability will be important in awarding these prizes.
- There are subcategory bonus prizes (5 prizes totaling $50,000) based on performance on specific subsets of the data
- The Explainability Bonus Track (5 prizes totaling $75,000) will be based on different evaluation criteria from the other prizes. These details are still being finalized—please hang tight.
We hope the opportunities to win the many prizes described above are still motivating for you!
Hi @jayqi,
Is it going to be possible to get feedback on our previous model reports, in order to improve on weak points for the final report?
Hi @mmiron,
Thanks for letting us know that you would like feedback. I will pass this along to the challenge organizers, but there are currently no plans to provide any feedback from the Hindcast model reports.
Hi @jayqi,
I saw two different types of files on the download page from Final Stage: prior_historical_label and cross_validation_label. Is it the same with the data from the Forecast Stage? Can I use the existing one for LOOCV training?
Hi @rasyidstat,
Yes, the two new files cross_validation_labels.csv and prior_historical_labels.csv should contain the same data as what was provided in the Forecast Stage.
Our preference is that you use the new files for clarity. You should be able to concatenate them together if you want a big dataframe that contains the data for both.