# Why PyTorch complains about a CUDA version mismatch, and how to fix it

> Source: https://netray.co/troubleshoot/cuda-version-mismatch-pytorch
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: On-Prem AI Troubleshooting

## Short answer

PyTorch ships its own bundled CUDA runtime inside the wheel, so it never uses your system's CUDA toolkit (the one nvcc reports). The only number that matters is the driver's maximum supported CUDA version, shown top right in nvidia-smi output. Fix the mismatch by installing a torch wheel built for a CUDA version at or below that number, not by touching nvcc or the toolkit.

## Error Title

CUDA version mismatch between PyTorch and the system driver

## Error Signature

RuntimeError: The NVIDIA driver on your system is too old (found version 11000). Please update your GPU driver by downloading and installing a new version

## Alternate Signatures

- RuntimeError: CUDA error: no kernel image is available for execution on the device
- UserWarning: CUDA initialization: CUDA unknown error, this may be due to an incorrectly set up environment

## Category

installation

## Stack

- pytorch
- cuda
- nvidia-driver

## Affects

Any PyTorch install where pip pulled a CUDA 12.x wheel onto a machine with an older driver; common right after `pip install torch` with no version pin.

## Quick Fix

### Fix it in a few minutes

**Steps:**

- Run nvidia-smi and note the CUDA Version shown in the top right corner. That is the maximum CUDA runtime your driver supports.
- Go to pytorch.org's install matrix and pick a torch build whose CUDA version is equal to or lower than that number.
- Uninstall the current torch with pip uninstall torch torchvision torchaudio.
- Reinstall using the exact --index-url pytorch.org gives you for that CUDA version, e.g. pip install torch --index-url https://download.pytorch.org/whl/cu121.
- Verify with python -c "import torch; print(torch.cuda.is_available(), torch.version.cuda)".

## Symptoms

- torch.cuda.is_available() returns False even though nvidia-smi shows a GPU
- RuntimeError mentioning the driver is too old, with a specific found version number
- CUDA operations fail immediately on first .cuda() call, before any real compute happens
- The error appears right after a fresh pip install of torch with no CUDA version pinned

## Causes

### Installed torch wheel targets a newer CUDA runtime than the driver supports

**Likelihood:** most-common

**Explanation:** A plain pip install torch pulls the latest default wheel, which today usually targets CUDA 12.x. If the machine's driver predates that CUDA generation, the driver refuses to run the bundled runtime because it cannot guarantee compatibility, so torch throws the version error before doing any GPU work.

**Fix:** Check nvidia-smi's top right CUDA Version, then reinstall torch from the matching CUDA index (cu118, cu121, cu124, etc.) using the exact --index-url from pytorch.org, not a bare pip install.

**Commands:**

- nvidia-smi
- pip uninstall -y torch torchvision torchaudio
- pip install torch --index-url https://download.pytorch.org/whl/cu121

### Driver was never updated after a GPU or OS upgrade

**Likelihood:** common

**Explanation:** Servers imaged months or years ago often keep the original driver even after the OS or GPU hardware changes. New torch releases assume recent CUDA generations, so an old driver that was fine for last year's torch build stops satisfying this year's default wheel.

**Fix:** Update the NVIDIA driver package for your distro (apt, dnf, or the official .run installer) to the latest production branch, reboot, then reconfirm with nvidia-smi before reinstalling torch.

### Confusing nvcc's toolkit version with the driver's supported CUDA version

**Likelihood:** occasional

**Explanation:** nvcc --version reports the locally installed CUDA toolkit used for compiling custom kernels. It has nothing to do with which CUDA runtime PyTorch uses at inference time, because torch wheels vendor their own cudart, cublas, and cudnn. Engineers chase the wrong number and upgrade the toolkit instead of the driver.

**Fix:** Ignore nvcc --version for this problem entirely. Only nvidia-smi's CUDA Version field and torch.version.cuda matter for choosing a compatible wheel.

### Container image ships a newer CUDA toolkit than the host driver allows

**Likelihood:** rare

**Explanation:** Base images like nvidia/cuda:12.4-runtime bundle a CUDA runtime that still has to pass the host driver's compatibility check via NVIDIA's forward-compatibility layer. On a host with an old driver and no compat package installed, containers built for newer CUDA fail the same way.

**Fix:** Either update the host driver or install NVIDIA's CUDA forward-compatibility package on the host, and prefer base images pinned to a CUDA version the fleet's oldest driver supports.

## Diagnostics

### Check the driver's maximum supported CUDA version

**Command:** nvidia-smi

**Interpretation:** The CUDA Version field top right is the ceiling your driver supports. Any torch wheel built for a CUDA version at or below this number will work; anything higher triggers the driver-too-old error.

### Check the locally installed toolkit (usually irrelevant to torch)

**Command:** nvcc --version

**Interpretation:** This is the compiler toolkit version, only relevant if you are compiling CUDA extensions yourself. It has no bearing on which CUDA runtime a pip-installed torch wheel uses.

### Check which CUDA runtime torch itself was built against

**Command:** python -c "import torch; print(torch.__version__, torch.version.cuda)"

**Interpretation:** torch.version.cuda shows the bundled runtime version. Compare it against nvidia-smi's CUDA Version; if torch.version.cuda is higher, that is the mismatch causing the failure.

### Confirm the GPU is actually reachable after reinstalling

**Command:** python -c "import torch; print(torch.cuda.is_available())"

**Interpretation:** True confirms the fix worked. False after a matching reinstall usually points to a deeper driver install problem rather than a version mismatch.

## Prevention

- Pin the exact torch version and --index-url in requirements.txt so CI and production always resolve the same CUDA build.
- Track the driver version deployed across your GPU fleet and treat driver upgrades as a planned, tested change, not incidental.
- Before bumping torch in any project, check its CUDA requirement against the oldest driver version in your fleet.
- In Docker, pin the base image's CUDA version to something your least-updated host driver can run, and document the minimum required driver.

## When To Escalate

If you manage a fleet with inconsistent driver versions across nodes, or you are air-gapped and cannot pull new drivers on demand, this stops being a one-machine fix and becomes a driver standardization and image management problem worth solving at the platform level.

## Faq

### Do I need to match nvcc's CUDA version to my PyTorch version?

**Answer:** No. PyTorch wheels bundle their own CUDA runtime (cudart, cublas, cudnn), so the locally installed toolkit that nvcc reports is irrelevant for running pip-installed torch. nvcc only matters if you are compiling custom CUDA extensions yourself, such as FlashAttention or a custom kernel, where the toolkit version must align with your compiler and PyTorch's build.

### What does the CUDA Version field in nvidia-smi actually mean?

**Answer:** It is the maximum CUDA runtime version your currently installed driver supports, not a CUDA toolkit that is installed. Any application, including a PyTorch wheel, whose bundled CUDA runtime is at or below that number will run correctly. A higher bundled version triggers the driver-too-old error.

### Can I fix this without updating my GPU driver?

**Answer:** Yes, in most cases. Instead of upgrading the driver, reinstall torch using an older CUDA build (for example cu118 instead of cu124) that your current driver already supports. This is usually faster and lower-risk than a driver upgrade on shared infrastructure.

### Why did this suddenly appear after I upgraded transformers or vllm?

**Answer:** Upgrading a library like transformers or vllm often pulls in a newer torch as a dependency automatically, silently changing your CUDA requirement. Always check pip's install log for a torch version bump whenever an unrelated package upgrade breaks GPU detection.

## Search terms

cuda version mismatch pytorch, nvidia driver too old for cuda version, torch.version.cuda, nvidia-smi vs nvcc, pytorch cuda compatibility, torch cuda is available false, pytorch cuda install wheel

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json