Failed to get convolution algorithm.
Tensorflow | allow_growth| cuDNN failed to initialize| Image classification
We get this error for image classification using TensorFlow on Windows PC with Nvidia . The solution led us to explore some options regarding GPU memory utilization.
Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node sequential_4/conv2d_12/Relu (defined at <ipython-input-98–471bb77471a2>:2) ]] [Op:__inference_train_function_3889]
The TensorFlow by default allocates full GPU memory even for a small network. One can set a fraction of GPU memory or allow growing memory usage based on need when constructing the session
Setting fraction of memory
# Assume that you have 12GB of GPU memory and want to allocate ~4GB:
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.333)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))The per_process_gpu_memory_fraction acts as a hard upper bound on the amount of GPU memory
Allowing growth of memory usage
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
session = tf.Session(config=config, ...)Packages list
Python
Python 3.8.8 64 bit
List
tensorboard 2.4.1
tensorboard-plugin-wit 1.8.0
tensorflow-estimator 2.4.0
tensorflow-gpu 2.4.1
termcolor 1.1.0
CUDA
CUDA 11.0 (cuda_11.0.3_451.82_win10) and cudnn-11.0-windows-x64-v8.0.4.30
Solution
Set the up allow_growth = True. I paste the following snippet in the begenning of the code.
from tensorflow.compat.v1 import ConfigProto
from tensorflow.compat.v1 import InteractiveSession
config = ConfigProto()
config.gpu_options.allow_growth = True
session = InteractiveSession(config=config)
References
