Preparing Windows for Tensorflow GPU training

There are some important requirements and dependencies which have to be exactly followed. In this article it is described how to use Tensorflow 2.1 with the respective CUDA 10.1 version. This setup most likely changes for future versions of Tensorflow. Check the Tensorflow websites for installation and gpu support to get an updated list of requirements.

Setup

• Windows 10
• Check CUDA support for your Nvidia GPU
• Python 3.7 64-bit version
• pip 19.0 or later
• Latest Microsoft Visual C++ Redistributable (link)

Installation

• Donwload the CUDA toolkit version 10.1 update2 for Windows 10 on the Nvidia website. Use either the network or local installer exe.
• For downloading the cuDNN library a Nvidia account is required. Create one and afterwards download the latest 7.6 version available. Be aware to select the cuDNN version compatible for CUDA 10.1.
• Install the CUDA 10.1 toolkit with the installer. Express installation works fine.
• Unzip the cuDNN library into an adequate directory. I choose C:\cuDNN\cuda101, as one could have different cuDNN versions on the same machine.
• Add the following paths to the environment variable PATH:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\extras\CUPTI\lib64
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include
C:\cuDNN\cuda101\bin

Test

If the above installation was completed successfully, setup a Tensorflow project and run the test script:

• open a terminal (e.g., powershell) and create an arbitrary project directory
• create a virtual environment python -m venv venv
• activate the environment venv\Scripts\activate
• install Tensorflow pip install tensorflow
• create a file with the script listed below inside the file test_tf_gpu_support.py and execute it via python test_tf_gpu_support.py:

import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))

The following is a successful output of the above script:

python .\test_tf_gpu_support.py
2020-03-18 08:54:34.836363: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-03-18 08:54:36.648195: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2020-03-18 08:54:36.776401: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.77GHz coreCount: 68 deviceMemorySize: 11.00GiB deviceMemoryBandwidth: 573.69GiB/s
2020-03-18 08:54:36.786815: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-03-18 08:54:36.799854: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
2020-03-18 08:54:36.806445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll
2020-03-18 08:54:36.811113: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll
2020-03-18 08:54:36.823310: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll
2020-03-18 08:54:36.830952: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll
2020-03-18 08:54:36.844950: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2020-03-18 08:54:36.849580: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]