Can I stop all processes using CUDA in Linux without rebooting? [closed]
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Improve this questionOriginal close reason(s) were not resolved
Is it possible to stop all running processing using the GPU via CUDA, without restarting the machine?
The lsof utility will help with this. You can get a list of processes accessing your NVIDIA cards with:
lsof /dev/nvidia*
Then use kill or pkill to terminate the processes you want. Note that you may not want to kill X if it's running. On my desktop system, both X and kwin are also accessing the GPU.
Long answer:
lsof /dev/nvidia*
gives you PIDs running on your GPU card which looks something like: lsof: status error on PID: No such file or directory
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python 7215 ******* mem CHR 195,255 434 /dev/nvidiactl
python 7215 ******* mem CHR 195,0 435 /dev/nvidia0
and
awk '{print $2}'
selects the PID column (in my case it is the second column) and
xargs -I {} kill {}
kills those PID jobs.
Short answer:
You may use the following command to remove them all at once.
Watch out! This command will delete all PIDs showing up for lsof /dev/nvidia*. Do run lsof /dev/nvidia* first to confirm these jobs are the ones you want to delete.
lsof /dev/nvidia* | awk '{print $2}' | xargs -I {} kill {}
Finish the job by a single command.
you can check the processes with nvidia-smi
and then
kill -9 <pid>
You can use the fuser command to get the all the processes using CUDA and then kill them. There's also a nice single command to kill them all.
sudo fuser -k /dev/nvidia*
精彩评论