hey guys!!
Can we use dev set for training the model as well apart from the evaluation?
Thank you
Of course.
The reason they give us a validation set, is to have validation set not only balance with test set in % of labels(0 or 1) also in the type of multi-modal hate speech.
We construct a dev and test set from 5% and 10% of the data respectively, and set aside the rest
to serve as fine-tuning training data. The dev and test set are fully balanced, and are comprised of
memes using the following percentages: 40% multimodal hate, 10% unimodal hate, 20% benign text
confounder, 20% benign image confounder, 10% random non-hateful.
Dev data is not in test. You can use it as validation and then you can make a final training for submission using train and dev data. You can also create your own split for validation, but be aware that then, it has not to be balanced with test set in the % of the type of multi-modal hate speech.
1 Like