Tensorflow-gpu issues

Product Help Packages & Environments
1

When I try to create a new python 3.7 or 3.8 (haven’t tried yet with 3.9 or 3.10) environment with the tensorflow-gpu 2.5.0 package installed, and I import tensorflow, I get the error “Could not load dynamic library ‘cudart64_110.dll’; dlerror: cudart64_110.dll not found”. I realize I can still use tensorflow, but I specifically chose the tensorflow-gpu package to have GPU support.

However, I have an existing python 3.7 environment that also has tensorflow-gpu 2.5.0 installed, and it is able to find the library OK. Furthermore, in this environment, if I run tf.test.is_gpu_enabled(), it is able to find my GPU correctly. Also, if I use Anaconda Navigator to clone this python 3.7 environment where I’m able to load the library, it also loads in the cloned environment. Yet, if I try to create a new environment from scratch, I get the error.

I’m not sure why I’m able to load the library in the one environment, but not the others, since the library is in the same location in each of the environments, and it should be a link back to the same file in the package cache, anyhow.

In the python 3.7 environment where I am able to load the cudart64_110.dll, the following relevant packages are installed:

# Name                    Version                   Build  Channel
cudatoolkit               11.3.1               h280eb24_9    conda-forge
cudnn                     8.2.1.32             h754d62a_0    conda-forge
python                    3.7.12          h7840368_100_cpython    conda-forge
tensorflow                2.5.0           gpu_py37h23de114_0
tensorflow-base           2.5.0           gpu_py37hb3da07e_0
tensorflow-gpu            2.5.0                h17022bd_0

In the python 3.8 environment where I’m not able to load cudart64_110.dll, the following relevant packages are installed:

# Name                    Version                   Build  Channel
cudatoolkit               11.3.1               h280eb24_9    conda-forge
cudnn                     8.2.1.32             h754d62a_0    conda-forge
python                    3.8.12          h7840368_2_cpython    conda-forge
tensorflow                2.5.0           gpu_py38h8e8c102_0
tensorflow-base           2.5.0           gpu_py38hb3da07e_0
tensorflow-gpu            2.5.0                h17022bd_0

Note that both environments include the same cudatoolkit version.

Also, I do realize that I’m mixing channels. However, (a) tensorflow 2.x is not available from conda-forge, and (b) that shouldn’t matter in this case since I am able to load tensorflow with CUDA in one environment, but not the other.

I asked this question on Stack Overflow, and I got a response saying that tensorflow 2.5.0 - 2.7.0 should be using CUDA 11.2 rather than CUDA 11.3, and referenced this page as proof.

However, that does not explain how I’m able to get tensorflow load the cudart64_110.dll library and see my GPU in the one environment where it does, despite having CUDA 11.3 installed there.

1 Like