Fix Tensorflow (GPU version) on Ubuntu 16.04

2016-08-30
#techtips

Tensorflow will not be able to load cudnn library on Ubuntu 16.04 if you simply install cudnn and cuda on you computer and you will get this:

1
2
3
4
5
6
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:99] Couldn't open CUDA library libcudnn.so. LD_LIBRARY_PATH: 
I tensorflow/stream_executor/cuda/cuda_dnn.cc:1562] Unable to load cuDNN DSO
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally

my GPU is a GeForce 730M chip with driver version 370.28, and I tried some other ways like modifying the .bash_profile

1
2
3
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
export CUDA_HOME=/usr/local/cuda
export PATH=/usr/local/cuda/bin:$PATH

only this one works (though it is not a clean installation perhaps…)

The issue according to orionr on nvidia forum is that TensorFlow expected all the CUDA and cuDNN libraries and headers to be installed under /usr/local/cuda. To deal with that you have to create some temporarily symlinks, which is ugly…

1
2
3
4
5
6
7
8
9
10
11
# Put symlinks in /usr/local/cuda
sudo mkdir /usr/local/cuda
cd /usr/local/cuda
sudo ln -s  /usr/lib/x86_64-linux-gnu/ lib64
sudo ln -s  /usr/include/ include
sudo ln -s  /usr/bin/ bin
sudo ln -s  /usr/lib/x86_64-linux-gnu/ nvvm
sudo mkdir -p extras/CUPTI
cd extras/CUPTI
sudo ln -s  /usr/lib/x86_64-linux-gnu/ lib64
sudo ln -s  /usr/include/ include

Register and nvidia developer account and download cuDNN’s binary build (not .deb) at here

Copy the files:

1
2
3
4
cd folder/extracted/contents
sudo cp -P include/cudnn.h /usr/include
sudo cp -P lib64/libcudnn* /usr/lib/x86_64-linux-gnu/
sudo chmod a+r /usr/lib/x86_64-linux-gnu/libcudnn*

Then install TensorFlow following its official instructions.

1
2
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.11.0rc0-cp27-none-linux_x86_64.whl
pip install --ignore-installed --upgrade $TF_BINARY_URL

And then it finally works!