Hi all-- we’ll go more into detail on this during the webinar next week, but for anyone who wants an early start I wanted to quickly review the DP Prescreen process.
The goal of the challenge is to develop an algorithm that provides good utility on the temporal map problem… while preserving privacy. The leaderboard metric only scores utility, which means the easiest way to get a high score is just to take the ground truth data, aggregate it into Neighborhood/Months, and submit it right back again with no privacy protection. Which is fair game for the open arena leaderboard, but not really to the objective of the challenge.
We also need to ensure your approach satisfies Differential Privacy. The first step to doing that is, we ask that you submit a write-up explaining your algorithm and proving that it satisfies differential privacy.
So! For your convenience–
- Resources explaining differential privacy are here
- Directions for prescreening are here
- Prescreen submissions are accepted here
- And a sample submission is in the competitor’s pack, here
SME Panel Prescreening happens every wednesday morning. If you submit by tuesday night, we’ll take a look wednesday and get you feedback promptly. Prescreening is a high-level check for obvious/significant mistakes. We also do a DP Validation review during final scoring, which involves a much closer read of the proof as well as a source code review. Naturally, only valid differentially private solutions are eligible for final prizes.
Prescreening is important for fun and profit! Once you’ve passed prescreening, you’ll be cordially invited to submit to a second, prescreened-only leaderboard. All submissions in the prescreened arena rank above all submissions on the open arena board. The easiest way to win a $1K progressive prize on Oct 23rd is to make sure you’ve passed prescreening by Oct 21st, and then enter the prescreened arena and submit to the leaderboard there.
Note that you must enter the prescreened arena before the end of the sprint on November 9th, in order to be eligible to be invited to final scoring.
If you’re new to differential privacy, it’s a good idea to submit to prescreening early, and then (optionally) resubmit each time you make a significant algorithm change, to confirm you’re continuing to satisfy differential privacy and to better ensure you’ll be eligible for final prizes. We’ll give you feedback and help you understand any oversights or violations. Welcome to the challenge!