Why PyTorch throws a cuDNN version error, and how to fix it

PyTorch wheels already include a specific cuDNN version compiled and tested against that exact torch release. When a different cuDNN version is separately installed system-wide, for example copied manually into /usr/local/cuda, and its directory appears in LD_LIBRARY_PATH ahead of torch's bundled path, the dynamic linker loads the system version instead, which was never validated against that torch build.

Fix: Remove the manually installed cuDNN directory from LD_LIBRARY_PATH entirely and let torch load its own bundled cuDNN, which is almost always sufficient since torch does not require a separate system install.

echo $LD_LIBRARY_PATH
python -c "import torch; print(torch.backends.cudnn.version())"

Conda environments and pip virtual environments can each install their own cudnn package independently. If a shell session accidentally activates the wrong environment, or a script is launched with an inherited environment variable pointing at a different environment's cuDNN, the version torch expects and the version actually loaded diverge silently.

Fix: Deactivate all environments and start from a clean shell, then activate only the single intended environment, verifying with which python and pip show torch before running the script again.

Some older tutorials instruct users to manually download cuDNN from NVIDIA and copy its header and library files directly into the CUDA toolkit's include and lib64 directories. This global system change affects every CUDA application on the machine, including ones like torch that never needed it, and any mismatch between this manually installed version and what torch expects produces the incompatibility error.

Fix: Remove the manually copied cuDNN files from the CUDA toolkit directory unless another application on the same machine specifically requires them, since torch itself does not need a system cuDNN install.

If TensorFlow, JAX, or another CUDA-aware library is imported before torch in the same Python process and that library dynamically loads its own bundled or system cuDNN first, torch's subsequent attempt to initialize CUDA operations can conflict with the already-loaded library version in memory.

Fix: Avoid importing multiple heavy CUDA frameworks in the same process where possible, or explicitly control import order and verify each framework's expected cuDNN version is compatible before combining them.

A manual cuDNN install that copies the versioned library files (e.g. libcudnn.so.8.9.0) but omits creating the standard unversioned symlink (libcudnn.so) that some tools expect can produce a missing library error even though the actual file exists on disk somewhere.

Fix: Recreate the expected symlinks pointing to the actual versioned library file, or simply remove the partial manual install and rely on torch's bundled cuDNN instead.