Are self-supervised or pseudo-label on test data allowed

Kelp Wanted: Segmenting Kelp Forests
1

Are any automatic learning on the test data allowed?
e.g. are self-supervised (e.g. masked autoencoder) or pseudo-label on test data allowed ?

1 Like
2

Hi @hengcherkeng,

Training on the test set is not allowed in any form. This includes but is not limited to: unsupervised parameter fitting (like PCA), self-supervised learning, manual annotations, pseudo-labeling.

See the below excerpts from the competition rules.

Entry submissions section:

Unless otherwise specified on the Competition Website, for the purposes of quantitative evaluation of Submissions, Participants agree to process each test data sample independently without the use of information from other cases in the test set. By default, this precludes using information gathered across multiple test samples during training, for instance through pseudo labeling. Eligible Submissions and models must be able to run inference on new test data automatically, without retraining the model.

Data use and code sharing

  • Participants may not add any manual annotations to the provided test data. Eligible solutions need to be able to run on test samples automatically using the test data as provided.