# How to pick which submission is used as the final one

**URL:** <https://community.drivendata.org/t/how-to-pick-which-submission-is-used-as-the-final-one/10990>\
**Category:** Youth Mental Health: Automated Abstraction\
**Created:** [November 19, 2024, 7:55pm UTC](https://community.drivendata.org/t/how-to-pick-which-submission-is-used-as-the-final-one/10990 "2024-11-19T19:55:33Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![aiva00](https://avatars.discourse-cdn.com/v4/letter/a/ac91a4/32.png) [@aiva00](https://community.drivendata.org/u/aiva00)\
**Post date:** [November 19, 2024, 7:55pm UTC](https://community.drivendata.org/t/how-to-pick-which-submission-is-used-as-the-final-one/10990/1 "2024-11-19T19:55:33Z")

</div>

Hey team,

I was wondering how can we select which out of all the submissions we have is going to be considered for the finale? I don’t think the score on the small sample of the hidden test set is very indicative of the performance on the full dataset.

Thanks

---

<div class="post-metadata">

**Author:** ![kwetstone](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@kwetstone](https://community.drivendata.org/u/kwetstone)\
**Post date:** [November 19, 2024, 8:36pm UTC](https://community.drivendata.org/t/how-to-pick-which-submission-is-used-as-the-final-one/10990/2 "2024-11-19T20:36:52Z")

</div>

@aiva00 Your best-scoring submission will automatically be selected and used for the final leaderboard. Good luck!

---

<div class="post-metadata">

**Author:** ![aiva00](https://avatars.discourse-cdn.com/v4/letter/a/ac91a4/32.png) [@aiva00](https://community.drivendata.org/u/aiva00)\
**Post date:** [November 19, 2024, 9:48pm UTC](https://community.drivendata.org/t/how-to-pick-which-submission-is-used-as-the-final-one/10990/3 "2024-11-19T21:48:02Z")

</div>

But whatif we don’t want the best scoring submission to be our submission? There is the possibility of overfitting to this small test set and maybe we want to select a more robust submission. Is that an option? @kwetstone

---

<div class="post-metadata">

**Author:** ![ishashah](https://avatars.discourse-cdn.com/v4/letter/i/f475e1/32.png) [@ishashah](https://community.drivendata.org/u/ishashah)\
**Post date:** [November 20, 2024, 1:56am UTC](https://community.drivendata.org/t/how-to-pick-which-submission-is-used-as-the-final-one/10990/4 "2024-11-20T01:56:23Z")

</div>

Hi @aiva00 , the submission that is automatically selected will be the one that performs best on the prize-determining leaderboard (not the public leaderboard).

---

<div class="post-metadata">

**Author:** ![sparsh35](https://avatars.discourse-cdn.com/v4/letter/s/4da419/32.png) [@sparsh35](https://community.drivendata.org/u/sparsh35)\
**Post date:** [November 20, 2024, 4:22am UTC](https://community.drivendata.org/t/how-to-pick-which-submission-is-used-as-the-final-one/10990/5 "2024-11-20T04:22:47Z")

</div>

True black-box testing relies on genuinely hidden test data to ensure meaningful and robust solutions. Allowing all submissions to the leaderboard might encourage brute-force optimization over rigorous validation. For instance, if the data has noise and the theoretical accuracy cap is 88%, the leaderboard might saturate around 86-87%. In such cases, the winner might simply be the participant with the most submissions, exploiting training stochasticity—an arguably flawed setup, in my humble opinion.