# Hardware Selection

**URL:** https://community.drivendata.org/t/hardware-selection/5014
**Category:** Genetic Engineering Attribution
**Created:** [September 7, 2020, 7:08am UTC](https://community.drivendata.org/t/hardware-selection/5014 "2020-09-07T07:08:07Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![chaits24](https://avatars.discourse-cdn.com/v4/letter/c/b19c9b/32.png) [@chaits24](https://community.drivendata.org/u/chaits24)
#### Post date: [September 7, 2020, 7:08am UTC](https://community.drivendata.org/t/hardware-selection/5014/1 "2020-09-07T07:08:07Z")

</div>

Hi everyone ,

Hope you are all doing well! I needed a suggestion related to hardware to run the ML algorithms on such a huge dataset. I was trying tree based algorithms on data using 8GB and also 12GB RAM hardware and still getting memory error every time I run the code.

Could anyone please guide on the configuration of hardware required or if there is any cloud platform where we can access high configuration hardware?

Thank you.

---

<div class="post-metadata">

### Author: ![xiaomen](https://avatars.discourse-cdn.com/v4/letter/x/7cd45c/32.png) [@xiaomen](https://community.drivendata.org/u/xiaomen)
#### Post date: [September 7, 2020, 9:01pm UTC](https://community.drivendata.org/t/hardware-selection/5014/2 "2020-09-07T21:01:21Z")

</div>

Hello @chaits24,

I do not think hardware is an issue here.  
My hardware:  
CPU: i7-4770  
RAM: 32GB  
GPU : gtx-1070

I could reach 0.91 on leaderboard with the following time to result:  
(from raw csvs to submission)

- without GPU : 3 to 4 hours
- with GPU : 20 mins

I would recommend 16GB at least though, and use chunking and iterator to avoid all in memory.

---

<div class="post-metadata">

### Author: ![hengcherkeng](https://avatars.discourse-cdn.com/v4/letter/h/85e7bf/32.png) [@hengcherkeng](https://community.drivendata.org/u/hengcherkeng)
#### Post date: [September 9, 2020, 2:42am UTC](https://community.drivendata.org/t/hardware-selection/5014/3 "2020-09-09T02:42:35Z")

</div>

" I could reach 0.91 on leaderboard with the following time to result … with GPU : 20 mins"

i would say that is impressive results. good work!

---

<div class="post-metadata">

### Author: ![chaits24](https://avatars.discourse-cdn.com/v4/letter/c/b19c9b/32.png) [@chaits24](https://community.drivendata.org/u/chaits24)
#### Post date: [September 9, 2020, 5:41am UTC](https://community.drivendata.org/t/hardware-selection/5014/4 "2020-09-09T05:41:46Z")

</div>

Those are very good results. Do you have any suggestions for cloud based platform that can provide 16GB or higher memory?

---

<div class="post-metadata">

### Author: ![KieranLitschel](https://avatars.discourse-cdn.com/v4/letter/k/3da27b/32.png) [@KieranLitschel](https://community.drivendata.org/u/KieranLitschel)
#### Post date: [September 10, 2020, 8:46am UTC](https://community.drivendata.org/t/hardware-selection/5014/5 "2020-09-10T08:46:09Z")

</div>

Google Colab is really good, and it’s free. Only 12GB of RAM though, but if you pay $10 a month you get access to higher memory (24GB) VM’s too. I’ve been using the free tier (0.8784 on the leaderboard atm) and found 12GB of RAM is enough. I did run into issues running out of memory as you have. I’d focus on trying to work out if there are any hyperparameters you can change to reduce how much memory the algorithm requires. If not then think about using a different algorithm.
