Could not select device driver "" with capabilities: [[gpu]]

Where's Whale-do?
1

Hi, thanks for running this competition. I’ve just forked the repository and run the suggested make commands both locally on a macbook, and on a server with a GPU. Everything runs smoothly for the CPU version, but when I get to make-test-submission on the GPU server, I get:

➜  make test-submission

chmod -R 0777 submission/
docker run \
        -it \
        --gpus all \
         \
        --network none \
        --mount type=bind,source="<repo-path>"/data,target=/code_execution/data,readonly \
        --mount type=bind,source="<repo-path>"/submission,target=/code_execution/submission \
        --shm-size 8g \
        --name belugas-competition \
        --rm \
        2cc32eca98e4
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].
Makefile:119: recipe for target 'test-submission' failed
make: *** [test-submission] Error 125

I tried on an alternative machine with a GPU as well and received the same message.

In case it’s useful:

➜  nvidia-smi 
Wed Jun  8 11:05:21 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.91.03    Driver Version: 460.91.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 208...  Off  | 00000000:43:00.0 Off |                  N/A |
| 32%   36C    P0    51W / 250W |      0MiB / 11019MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  GeForce RTX 208...  Off  | 00000000:44:00.0 Off |                  N/A |
| 33%   34C    P0    42W / 250W |      0MiB / 11019MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Thanks for any help with this. I don’t have much experience with Docker, which is part of the appeal of giving this challenge a go :smile:

Oliver

2

solved. To use docker on a machine with a GPU I needed to install nvidia-docker GitHub - NVIDIA/nvidia-docker: Build and run Docker containers leveraging NVIDIA GPUs

I also had to change permissions of the data directory (chmod -R 777 data/) and the produced submissions.csv, but I’m assuming that this is a problem with my system. I’m making a note of it though in case anyone else experiences the same thing.

3 Likes