Publishing following repository that contributes to 6th place solution
Go through many reid papers and code implementation but to me sub center arcface loss worked.
Waiting for blog solution to learn and improve upon.
Regards
Aditya
1 Like
Thank you for publishing this! I have used your training notebook to train up a model that gets a ~0.4 score, which is better than the online score I achieved on my own, although I think my NN code is not very dissimilar to the architecture you use (I used an embedding length of 768, I used FastAI, etc.)
I am curious about this code in your val_score function:
pred_dfs['tmp'] = pred_dfs.query_id.apply(lambda x: x.split('-')[-1])
pred_dfs = pred_dfs[pred_dfs.database_image_id != pred_dfs.tmp]
pred_dfs = pred_dfs.reset_index(drop=True)
del pred_dfs['tmp']
In the 8 validation scenarios you provide, the length of pred_dfs doesn’t change over this block. I think I understand what the intention of this code block is, but I don’t want to put words in your mouth. Can you explain the purpose? Do you have any sense if this block was important in the private LB scenarios?
I have been struggling to understand why my local MAP scores were very different from the cloud-scored results. When I compare your model (trained on my system) to my model, I get results that seem quite similar to yours (although mine does much worse on scenario 6):
| Scenario | Yours | Mine |
|---|---|---|
| Scenario 0 | .441 | .523 |
| Scenario 1 | .608 | .567 |
| Scenario 2 | .587 | .626 |
| Scenario 3 | .482 | .582 |
| Scenario 4 | .315 | .393 |
| Scenario 5 | .316 | .526 |
| Scenario 6 | .240 | .085 (!) |
| Scenario 7 | .055 | .032 |
| Overall | .412 | .445 |
These similar scores lead me to suspect that I was making a mistake in either filtering or preparing the data for submission.
Nothing much. Removing query image, if any, from database images.
Like you I also struggled a lot with CV, so I tried to create this type of validation set by simulating as much details as mentioned in Problem Description.
Due to not much submission left, I have not tried single fold submission but overall cv matched with competition metric.
Like you I also suspected submission code, but the bug(?) I found was in creating validation set (training set whale ids are in validation set with time difference between them was low)
May be winner can also describe their validation approach other than their modeling approaches.
In Drivendata competition , for me creating robust CV mechanism is difficult. So there is always learning which I think is very much required to tackle real life problems.