Hello PETs Prize Phase 2 participants,
Challenge organizers are considering a possible addition to the evaluation API to give more flexibility to teams. We are interested in hearing whether this addition would be helpful to teams. Draft documentation is below.
—
Setup Functions
An optional setup step will be run before federated training and federated inference. You can make use of this step by including train_setup and/or test_setup functions in your solution_federated.py module. The setup functions are intended to make it simpler to perform initial setup between the parties, by avoiding the need for explicit communication routed via the server. For example, a setup function might generate key pairs and initialize each client’s state with the public keys of the other parties, or initialize each client’s state with a shared symmetric key.
The computation cost of the setup functions will be measured during evaluation, but their communications overhead will not be measured (because the communication they encode is not explicit). The setup function should therefore not implicitly encode significant communications between the parties.
If you use a setup function, your technical report should describe exactly what computation it performs, and what communications it encodes. Reviewers and judges will be instructed to take this description into account when evaluating the performance of your submission—for example, by reducing performance scores if the setup function has been used to hide communications costs that should have been measured during evaluation. This setup step is not provided explicit access to training or test data and should not perform any computation using the data.
The train_setup and test_setup functions should have the following function signatures.
def train_setup(server_dir: Path, client_dirs_dict: Dict[str, Path]):
"""
Perform initial setup between parties before federated training.
Args:
server_dir (Path): Path to a directory specific to the server/aggregator
that is available over the simulation. The server can use this
directory for saving and reloading server state. Using this
directory is required for the trained model to be persisted between
training and test stages.
client_dirs_dict (Dict[str, Path]): Dictionary of paths to the directories
specific to each client that is available over the simulation. Clients
can use these directory for saving and reloading client state. This
dictionary is keyed by the client ID.
"""
...
def test_setup(server_dir: Path, client_dirs_dict: Dict[str, Path]):
"""
Perform initial setup between parties before federated test inference.
"""
...