Transfer learning: What is ResNet learning?

[DavidEis]

Dear Conser-visionists,

ML / Deep Learning newbee here I’ve tried several approaches but couldn’t get any improvements over the benchmark results.
To better understand what’s going on, I used the great benchmark script and adapted the fraction of training images and epochs of training to let the model train on different numbers of images.
Below, I’ve plotted the training loss as well as the public loss for some configurations - Training loss goes down as expected, but public loss is increasing.

So it seems that while the model is learning to distinguish training data incl. unseen validation data in this set, there is no correlation to the test data.

Any thoughts on this behaviour and how the model can actually learn the “right” things?

Best regards,
David

[steve_shooty]

Hy DavidEis,
better later as never
What you see is a typical example of overfitting. That means that your model while training (over-)performed about your training data. What you need is generalization (next buzz-word) that means that your model have a good accuracy on ‘new’ unknown data.
You can monitor it with a dev set that you split out of your training set before you train it with these data. But look at the distribution of classifications (this must have the same as the train set after splitting). Watch on the best accuracy of the dev set, not the train set.
I hope it was a little help to the right way. Use search machine to read some examples about this theme.

Have fun,
steve_shooty