Assertionerror: invalid device ID and runtimeerror: CUDA error: invalid device ordinal

I am using torch These two problems occur when multiple cards are parallel .

## Question 1 ：AssertionError: Invalid device id, Invalid device id

The reason for this ： Combined with code interpretation ：

import ...

os.environ["CUDA_VISIBLE_DEVICES"] = "2,3"


model = model(...)

torch.cuda.set_device(2)
model = torch.nn.DataParrel(model, device_ids=[2,3])

The reason for the error ：os.environ["CUDA_VISIBLE_DEVICES"] = "2,3" This statement will the original divice：2 and device：3 The number of is mapped to device：0 and device：1, So in set_device Error reporting at time , Invalid device id

terms of settlement ： Method 1 ： take os.environ Delete . If some cards are occupied, you have to use os.environ To set up available devices , Use method 2 ： Follow the remapped number , example set_device（0）

## Question two ：RuntimeError: CUDA error: invalid device ordinal

The reason for this ： Combining with the code

import ...

os.environ["CUDA_VISIBLE_DEVICES"] = "2,3"


model = model(...)


model = torch.nn.DataParrel(model, device_ids=[2,3])

The reason for the error ： Same as above ,os.environ["CUDA_VISIBLE_DEVICES"] = "2,3" This statement will the original divice：2 and device：3 The number of is mapped to device：0 and device：1, So in execution torch.nn.DataParrel When an error

terms of settlement ： Method 1 ： take os.environ Delete . If some cards are occupied, you have to use os.environ To set up available devices , Use method 2 ： Follow the remapped number , example ：

model = torch.nn.DataParrel(model, device_ids=[0,1])

Recommend an article , Simple implementation of multi card parallel ：

https://muzhan.blog.csdn.net/article/details/109318226

https://www.codeleading.com/article/2345206500

https://blog.csdn.net/weixin_34233421/article/details/91396978