- The hindcast report is only required to be considered for the hindcast prizes (this seems clear) and not reviewed during evaluation for the overall prizes.
This is correct. The final model report will be reviewed for the overall prizes instead of the hindcast model report. The final model report is expected to be an expanded version of the hindcast model report.
- The Jan 11 forecast code submissions and March 15 report submissions can include changes to our models that we make after our final hindcast submission.
Yes, you will be allowed to make updates to your model in between the deadlines of the different stages. At the very least, additional training data will be released (the years that are in the Hindcast Stage holdout test set) that we expect you to use for training for the Forecast Stage and will be part of the cross-validation.
- The 30% of the evaluation criteria assigned to “Forecast Skill (Hindcast)” is not an evaluation of the December submitted hindcast code but rather a new submission that will include the mentioned cross-validation. And we are not required to use the same model we submitted in December.
Correct. The cross-validation results will supersede the Hindcast Stage performance for the overall prize evaluation.
- Evaluation of our final model report and explainability track report requires that we submitted hindcast code, but doesn’t require us to discuss it in the report.
In order for the judges to understand your methodology, the final report requirements may ask you to summarize the changes that you’ve made between the different stages in the challenge. However, as noted, the performance in the Hindcast Stage will not be considered.