Evaluation Metrics

PETs Prize Challenge
1

Hello,

In the submission guidelines “communication cost for each party” is suggested as an efficiency metric to measure performance. We understand there are other efficiency metrics defined e.g. total execution time, peak memory usage, network file/disc volume. Could you clarify what communication cost specifically would be?

Thanks,
Sarah

2

Hi @sarahrhalem,

In the simulated federated training and inference, we capture all client–server communications on disk. The file volume and disk size of the captured communications is the measure of network usage.

1 Like