Solutions postings

as this is my first driven data competition, I don’t know if the solutions are usually posted. I would love to see what zfturbo did to get twice my score!

I currently preparing solution write-up for organizers. I also plan to publish small solution writeup on arxiv after that (I will post link in this thread if you interested).

Code will be posted on drivendata github a little bit later, like it was in previous competitions.

My solution is based on these 2 libraries I prepared and already posted:

  1. The best model for me was DenseNet121 3D with (96, 128, 128, 3) input shape and batch size equal to 6. I used large dropout 0.5 at the classifcation layers to prevent overfitting. I started with imagenet weights which I converted for 3D variant of nets.
  2. I trained only on ROI part of videos, which I extracted using your code from forum: Python code to find the roi - #4 by Shanka ))
  3. Batches generated in proportion 25 (stalled==1) / 75 (stalled == 0)
  4. I validate using MCC from begining and then switched to ROC AUC. My validation score was around 0.96-0.98 ROC AUC.
  5. I started with micro dataset and then increase number of used videos up to ~50K (using all available stalled == 1)
  6. Last trick which allows me to increase score from 0.82 to 0.86 on public LB was to finetune only on tier1 data (looks like test set contains only tier1 ???)
  7. I applied augmmentations with volumentations library which I remade a little bit to increase speed and add some more useful augs.
  8. I used 5KFold cross-validation. My validation MCC score wasn’t the same as LB, but direction was similar. Increasing at local gives me better result on LB.
  9. Loss function: binary crossentropy. Optimizer: AdamAccumulate
  10. I choose THR to output binary probabilities using leaderboard (so there was some chance to overfit on LB). I found out the optimal number of stalled videos in test set was around 600-700.
1 Like

Thank you! I went the LSTM way, was probably the wrong decision.

Hi,
Congratulations on first place!

I hope you wouldn’t mind answering a few questions on your solution?
Whats the intuition behind using unbalanced batches? I’ve not seen that approach used before!
Did you/How did you decide on a threshold for the test dataset?
And how did you convert 2d imagenet weights to 3d weights? I’ve had a look at the code but I can’t work out what you’re doing! (I’m less familiar with keras).

Thanks :slight_smile:

Congratulations on first place!

Thank you )

Whats the intuition behind using unbalanced batches? I’ve not seen that approach used before!

In early stage of competition I found out that test set is very imbalanced. It has around 20 times more stalled=0 in it. So by generating inbalanced mini-batches I tried to make neural net more “optimistic”, e.g. predict “1” only if it really sure video has label stalled==1. I’d probably try 1 to 20 ration in batches, but I was limited by batch with size equal to 6, which I used. And I didn’t want to have “only zeros” batches during training.

Did you/How did you decide on a threshold for the test dataset?

In the begining I tried different THRs and check LB to find which is better. At the latest stages I choose THR to keep number of stalled=“1” in submission.csv in range ~600-700.

And how did you convert 2d imagenet weights to 3d weights? I’ve had a look at the code but I can’t work out what you’re doing! (I’m less familiar with keras).

My answer on this question was big enough that It’s better to read PDF I prepared.