Hello PETs Prize Challenge participants,
Welcome to the home stretch! Final submissions eligible for Phase 2 prizes are due by January 24, 2023 January 26, 2023 at 23:59:59 UTC . (Please note the time zone!) See below for information on making your final submission.
Deadline updated, please see this announcement.
Making Final Submissions
Final submissions are available on the challenge website. On the submission upload forms you’ve previously been using for submission testing, there is now a toggle switch for “Normal” or “Smoke test” submissions.
A “Normal” submission will be run on the final evaluation dataset. You are allowed up to 3 successful submissions, and will be required to choose 1 as the final submission to be considered by judges. Instructions for making the final selection will be discussed more below. “Smoke test” submissions will be run as test submissions on the 30% of development data as before, will not count for judging, and have a separate daily limit.
To select the 1 final submission for consideration by judges, you should look at the submission list page, which is navigable via the “Submissions” sidebar link. In the table showing your submissions with numerical PRAUC scores, you will see checkboxes for you to select your final submission.
What you need to submit
As documented on the Problem Description page for your track, you will need to submit the following:
-
Executable federated solution code—working implementation of your federated learning solution that is run via containerized execution.
- Submit your code via the federated submission form for your track.
- Make sure at least one submission is selected in the federated submissions list for your track.
-
Executable centralized solution code—working implementation of a centralized version of your solution that is run via containerized execution.
- Submit this via the federated submission form for your track
- Make sure at least one submission is selected in the federated submissions list for your track.
-
Documentation ZIP archive—a zip archive that contains the following documentation items: Technical report—updated paper that refines and expands on your Phase 1 concept paper, including your own experimental privacy, accuracy, and efficiency metrics for the two models; and Code guide—README files which provide the mapping between components of your solution methodology and the associated components in your submitted source code, to help reviewers identify which parts of your source code perform which parts of your solution.
- Submit this via the “Documentation Submission” form on the federated solution webpage for your track. You have the ability to delete a previous submission and resubmit if you need to make changes.
See the below table for direct links to each track’s submission forms.
| U.S. Track A: Financial Crime Prevention | U.S. Track B: Pandemic Forecasting | U.K. Track A: Financial Crime Prevention | U.K. Track B: Pandemic Forecasting | |
|---|---|---|---|---|
| 1. Executable federated solution code | - Submission form - Submission selection | - Submission form - Submission selection | - Submission form - Submission selection | - Submission form - Submission selection |
| 2. Executable centralized solution code | - Submission form - Submission selection | - Submission form - Submission selection | - Submission form - Submission selection | - Submission form - Submission selection |
| 3. Documentation ZIP archive | - Submission form | - Submission form | - Submission form | Submission form |
Deadline
The deadline for Phase 2 final submissions is January 24, 2023 January 26, 2023 at 23:59:59 UTC. (Please note the time zone!)
Deadline updated, please see this announcement.
A number of teams have asked regarding deadline extensions. There are no plans to extend the challenge deadline. Teams must submit all required materials by the deadline. Please note that the challenge timeline is constrained by the 2023 Summit for Democracy happening at the end of March.
Challenge organizers are available to help! If you need clarification about the rules of the challenge, or if you need help debugging the technical infrastructure, please reach out on the community forum, on the NIST Slack channel for the challenge, or via email to info@drivendata.org.
Please test your submissions as early as possible to give yourselves and challenge organizers time to work through any issues that you may encounter.