We got a 3-part question about how to submit Phase 3 solutions during the office hours event today. We are sharing a detailed response here @maryamzolnoori!
- Could you please clarify how we should submit the code? Is there a specific platform or website designated for this? Additionally, are there any formatting or structural requirements we should follow when publishing the code?
Upload your submission as a ZIP archive that follows the submission format via the “Submit” page.
- We understand that two documents are required for the report: one five-page document and another three-page document. Could you please clarify the difference in purpose and content between the two?
The submission requirements section describes the purpose and content of each of these documents, and provides a basic template for each in the “suggested sections and prompts” sections.
- Could you provide more details about the generalizability criteria you expect to see addressed in the report?
Here is what judges will consider when evaluating your model’s generalizability: “How likely is the model to generalize well to a real-world population of interest, in particular historically underserved segments of the population? This can be demonstrated through incorporation of new data, bias assessment and mitigation, and rigorous model performance assessment.”
Each team will need to show that their model generalizes to a real-world population of interest, but the best way to do this depends on the type of model and any new data sources used. Teams should review Phase 3 materials closely, and can also look at shared resources and guidance from Phases 2 and 1 for inspiration and to build a stronger sense for general challenge goals related to generalizability. Evaluation criteria and submission requirements from earlier phases do not apply to Phase 3 and will not inform evaluation, but the Phase 1 and 2 websites may have relevant tips and resources for understanding and demonstrating generalizability towards challenge goals.