# Resource issue when submitting for Track B federated **URL:** \ **Category:** PETs Prize Challenge\ **Created:** [January 23, 2023, 5:38pm UTC](https://community.drivendata.org/t/resource-issue-when-submitting-for-track-b-federated/8496 "2023-01-23T17:38:14Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![kzliu](https://avatars.discourse-cdn.com/v4/letter/k/d07c76/32.png) [@kzliu](https://community.drivendata.org/u/kzliu)\ **Post date:** [January 23, 2023, 5:38pm UTC](https://community.drivendata.org/t/resource-issue-when-submitting-for-track-b-federated/8496/1 "2023-01-23T17:38:15Z")
Hi, I’m running into the following errors with my federated submission for Track B Pandemic: ```auto ... Error: No available node types can fulfill resource request {'CPU': 5.0, 'GPU': 1.0}. Add suitable node types to this cluster to resolve this issue. ``` The submission is still ongoing for several hours now. Is the issue expected? In case this is due to a submission queue – if our solution have an option to not use an GPU, would it help speed up the evaluation? Thanks! ---
**Author:** ![jayqi](https://yyz2.discourse-cdn.com/flex028/user_avatar/community.drivendata.org/jayqi/32/924_2.png) [@jayqi](https://community.drivendata.org/u/jayqi)\ **Post date:** [January 23, 2023, 6:12pm UTC](https://community.drivendata.org/t/resource-issue-when-submitting-for-track-b-federated/8496/2 "2023-01-23T18:12:31Z")
Hi @kzliu, This is not related to the submission queue, and probably not related to whether your solution involves a GPU. The entire job runs inside one container. When it says “cluster” in the log, there is not actually any cluster; Ray is just doing multiprocessing on one node. This error log says that inside this container—which is supposed to have 6 CPUs and 1 GPU—it’s not able to spawn a new process that has access to 5 CPUs and 1 GPU. I’m not seeing this error for other running jobs, so there is something specific about your solution that makes the system think that there are not enough CPUs or GPUs available. Are there other process you’ve run from the server/strategy that are using significant resources? ---
**Author:** ![kzliu](https://avatars.discourse-cdn.com/v4/letter/k/d07c76/32.png) [@kzliu](https://community.drivendata.org/u/kzliu)\ **Post date:** [January 23, 2023, 6:17pm UTC](https://community.drivendata.org/t/resource-issue-when-submitting-for-track-b-federated/8496/3 "2023-01-23T18:17:58Z")
I see — I was hoping to try CPU only execution by putting `os.environ['CUDA_VISIBLE_DEVICES'] = ''` at the top of my solution files. Would this be relevant? If so would there be any other recommended way to disable GPUs? Thanks! ---
**Author:** ![jayqi](https://yyz2.discourse-cdn.com/flex028/user_avatar/community.drivendata.org/jayqi/32/924_2.png) [@jayqi](https://community.drivendata.org/u/jayqi)\ **Post date:** [January 23, 2023, 6:23pm UTC](https://community.drivendata.org/t/resource-issue-when-submitting-for-track-b-federated/8496/4 "2023-01-23T18:23:57Z")
If you set the environment variable `CPU_OR_GPU` to `"cpu"` (or in reality, any value that is not `"gpu"`), it should not request a GPU when spawning new processes for running Client methods. [This](https://github.com/drivendataorg/pets-prize-challenge-runtime/blob/d140e87d0bdc48607ac12809e5b7bbbdb9737859/runtime/main_federated_train.py#L80-L81) is the code where that configuration happens. ---
**Author:** ![jayqi](https://yyz2.discourse-cdn.com/flex028/user_avatar/community.drivendata.org/jayqi/32/924_2.png) [@jayqi](https://community.drivendata.org/u/jayqi)\ **Post date:** [January 23, 2023, 6:33pm UTC](https://community.drivendata.org/t/resource-issue-when-submitting-for-track-b-federated/8496/5 "2023-01-23T18:33:22Z")
Please also note that if you’re seeing that message at all, then that’s an error that likely won’t recover over time. You should just cancel that job in order to free up the shared resources for other participants. It may be the case that Ray is unable to spawn a new process because the number of _CPUs_ available is not enough. I’m taking a look at this to see if there is a fix I can make for this. ---
**Author:** ![kzliu](https://avatars.discourse-cdn.com/v4/letter/k/d07c76/32.png) [@kzliu](https://community.drivendata.org/u/kzliu)\ **Post date:** [January 23, 2023, 6:39pm UTC](https://community.drivendata.org/t/resource-issue-when-submitting-for-track-b-federated/8496/6 "2023-01-23T18:39:12Z")
I have cancelled the job and resubmitted another one with `os.environ['CPU_OR_GPU'] = 'cpu'` instead for now. Thanks for looking into this!