CUDA driver version is insufficient for CUDA runtime version

I got the mess开发者_Python百科age:

"cutilCheckMsg() CUTIL CUDA error :

kernel launch failure : CUDA driver

version is insufficient for CUDA

runtime version."

While trying to run an example source code. Also happens for the function cutilSafeCall.

I am using:

• Windows 7 64bits
• Visual studio 2008
• CUDA developer driver, toolkit, and SDK 3.1
• Emulation mode

You need to ensure that your driver version matches or exceeds your CUDA Toolkit version.

For 2.3 you need a 190.x driver, for 3.0 you need 195.x and for 3.1 you need 256.x (actually anything up to the next multiple of five is ok, e.g. 258.x for 3.1).

You can check your driver version by either running the deviceQueryDrv SDK sample or go into the NVIDIA Control Panel and choose System Information.

Download an updated driver from www.nvidia.com/drivers.

I saw the same at runtime with the latest driver on Mac OS 10.6.

cudaError_t error = cudaGetDevice(&device);
printf("%s\n", cudaGetErrorString(error));

I went back to the developer site, downloaded the driver again and now it runs. http://developer.nvidia.com/object/cuda_3_1_downloads.html#MacOS

You can either download the latest driver OR use an older toolkit version to compile your code.

Counterintuitively, this error also happens if libcuda.so is not found, even when versions reported by nvidia-smi match perfectly. This library is part of nvidia-drivers package (On CentOS: nvidia-driver-latest-cuda-libs, on Gentoo x11-drivers/nvidia-drivers). It is possible to have the CUDA Tookit with nvcc and libcudart installed and building your app fine, but the drivers part not installed, causing this error.

To diagnose whether this is the reason, use strace:

strace -f -e trace=file ./your_cuda_app

and check for open calls to libcuda.so*, at least one of them should return with a success code, like so:

4928  open("/lib64/libcuda.so.1", O_RDONLY|O_CLOEXEC) = 3 

My cent,

with Linux/Unix this error may be related to the selected GPU mode (Performance/Power Saving Mode), when you select (with nvidia-settings utiliy) the integrated Intel GPU and you execute the deviceQuery script... you get this error:

-> CUDA driver version is insufficient for CUDA runtime version

But this error is misleading, by selecting back the NVIDIA(Performance mode) with nvidia-settings utility the problem disappears.

It is not a version problem.

Regards

P.s: "Power Saving Mode" tells Optimus to activate the CPU integrated Intel GPU

CUDA driver version is insufficient for CUDA runtime version: means your GPU can`t been manipulated by the CUDA runtime API, so you need to update your driver.

In my case, I had to run my docker container with nvidia-docker run ... instead of docker run ...

I also had similar problem, updated my graphic driver but the problem still remained. I finally decided to remove Cuda 9.2 and install Cuda 8, it solved my issue.

This is what has worked for me :

1. Installing TensorFlow version 1.14, installing keras
2. Install CUDA 10.0 from https://developer.nvidia.com/cuda-10.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exenetwork for windows
3. Download the compatible CuDNN i.e 7.4.2 from https://developer.nvidia.com/rdp/cudnn-archive
4. set the following in the path variable cudnn-10.0-windows10-x64-v7.4.2.24\cuda\bin cudnn-10.0-windows10-x64-v7.4.2.24\cuda\include cudnn-10.0-windows10-x64-v7.4.2.24\cuda\lib\x64
5. Restart the Pc

This problem can also be because of incorrect environment setup, e.g. Docker image setup. Although the driver itself is correct, sufficient for your program. If your LD_LIBRARY_PATH points to the wrong driver, it can throw this error. In my case, i get this error when using /usr/local/nvidia/lib/libcuda.so, and if I use /usr/local/nvidia/lib64/libcuda.so everything goes right.

Maybe it is related to the TBB lib: Error OpenCV with CUDA using TBB for multiple GPUs

Try rebuilding it making sure you passed the following parameters to CMake (assuming you already installed "tbb" and "tbb-devel" packages:

-D WITH_TBB=YES -D TBB_INCLUDE_DIRS=/usr/include/tbb