Back to DrivenData | Blog

Some questions about the competition

TissueNet: Detect Lesions in Cervical Biopsies
#1

  1. Competition rules. The game rules are not stated anywhere in the game, but your default rules, which are unfair to newcomers to Drivendata.

  2. Two test sets. Since all submissions have results on the two validation sets A and B, the two test sets have the same weight, and the B set cannot achieve the result of preventing overfitting as same as A. On the contrary, according to result B, it is more like overfitting.

  3. The distribution gap between the two test sets A and B. I feel that the organizer has not done any pre-experiments, and the data set is divided without considering the data distribution, which is irresponsible to any participant.

I hope the organizers can reconsider the unfairness and unreasonableness in the competition.
Hope there is an explanation before the results are announced.

@glipstein @bull

#2

@AndrewTal @glipstein @bull
I very much agree with what you said. I originally thought that there was only one test set. I never mentioned an additional test set in the document. Moreover, this competition indicator is very sensitive to the data distribution, and his automatic submission is subject to the highest submission. Other teams with fewer submissions are unfair

1 Like
#3

Like the kaggle competition platform, you can only choose two times, but this competition from the beginning of the description to the final evaluation of the score is not fair. For example, some teams submit 8 times and take the highest 8 times, but many teams only have 4. Times, take the highest 4 times, which is very unfair

1 Like
#4

If the competition loses its fairness, then everything about it is meaningless. As mentioned above, there are extreme unfairness in the competition. The existing scoring rules (invisible) cannot judge the quality of the algorithm reasonably. The current results have failed all the participants’ efforts and trust in the competition. I sincerely hope that the organizer can consider a more reasonable ranking method.
@glipstein @bull