# Why vLLM installation fails, and how to get a clean working install > Source: https://netray.co/troubleshoot/vllm-installation-errors > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: On-Prem AI Troubleshooting ## Short answer vLLM ships prebuilt wheels compiled against a specific, exact torch and CUDA version, and it uses custom CUDA kernels that only work with that pairing. Most install failures happen because torch was already installed separately, or an existing environment has an incompatible CUDA toolkit, so the fix is almost always a fresh virtual environment where pip resolves vLLM and its exact torch dependency together in one pass. ## Error Title vLLM fails to install or import due to torch and CUDA mismatches ## Error Signature ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. vllm 0.6.3 requires torch==2.4.0, but you have torch 2.5.1 which is incompatible ## Alternate Signatures - ImportError: libcudart.so.12: cannot open shared object file: No such file or directory - RuntimeError: vLLM is not compatible with your CUDA version ## Category installation ## Stack - vllm - pytorch - cuda ## Affects vLLM across recent versions, especially when torch or another CUDA-dependent package is pre-installed in the environment before vLLM. ## Quick Fix ### Fastest path to a clean vLLM install **Steps:** - Create a fresh virtual environment: python -m venv vllm-env && source vllm-env/bin/activate. - Do not pre-install torch. Run pip install vllm and let it pull the exact torch version it requires. - If you must use a specific CUDA build, check vLLM's release notes for the supported CUDA version and install using the matching prebuilt wheel or index. - Verify the install with python -c "import vllm; print(vllm.__version__)". - Run a minimal offline inference smoke test before deploying a server. ## Symptoms - pip reports a dependency conflict naming an exact torch version vLLM requires - import vllm raises an ImportError about a missing shared CUDA library - vLLM installs cleanly but crashes on first model load with a CUDA kernel error - Installation works on one machine but fails on another with the same OS ## Causes ### torch was installed separately before vLLM, creating a version conflict **Likelihood:** most-common **Explanation:** vLLM pins an exact torch version per release because its custom CUDA kernels (for paged attention and continuous batching) are compiled against that specific torch build's ABI. If a different torch is already present, pip either refuses the install or silently leaves an incompatible pairing that crashes at runtime instead of at install time. **Fix:** Uninstall any existing torch first, or better, start from a fresh environment and let pip install vllm resolve the correct torch version itself rather than pre-installing torch manually. **Commands:** - pip uninstall -y torch torchvision torchaudio - pip install vllm ### Generic pip install pulled a wheel built for the wrong CUDA version **Likelihood:** common **Explanation:** vLLM publishes wheels tied to specific CUDA major versions. On a system whose driver only supports an older CUDA generation, the default wheel from PyPI may target a newer CUDA than the driver allows, producing the same driver-too-old failure seen with plain PyTorch installs. **Fix:** Check the vLLM documentation for the CUDA version matrix for your target release, and install using the CUDA-specific instructions (a matching prebuilt wheel or explicit index) rather than the bare pip install vllm command. ### Building vLLM from source without a matching CUDA toolkit **Likelihood:** common **Explanation:** Installing from source instead of a prebuilt wheel requires compiling custom CUDA kernels with nvcc, which means the installed CUDA toolkit version must align with what vLLM's build system expects. A mismatched or missing toolkit causes the compilation step itself to fail with cryptic nvcc errors. **Fix:** Prefer the prebuilt wheel whenever your platform is supported; only build from source when you need an unreleased feature, and match the CUDA toolkit version to vLLM's documented build requirements first. ### Unsupported Python version forces a slow source build with its own failures **Likelihood:** occasional **Explanation:** vLLM publishes prebuilt wheels for specific Python versions. A very new Python release (for example, right after a Python minor version launches) often lacks a prebuilt wheel for weeks, so pip silently falls back to building from source, which then hits unrelated compiler and toolchain errors. **Fix:** Check vLLM's supported Python versions in its documentation and use one with published wheels, typically by creating the virtual environment with an explicit, slightly older Python version such as 3.10 or 3.11. ### Conflicting xformers or flash-attn versions pulled in as transitive dependencies **Likelihood:** rare **Explanation:** vLLM depends on attention kernel libraries like xformers or FlashAttention for some backends. If an incompatible version is already installed, or a later manual pip install upgrades one of them independently, the mismatch surfaces as an import error deep inside vLLM's attention backend selection. **Fix:** Let vLLM's own dependency resolution manage xformers and flash-attn versions; avoid manually pinning or upgrading them unless you have confirmed compatibility in vLLM's release notes. ## Diagnostics ### Check what versions actually got installed **Command:** pip show vllm torch **Interpretation:** Compare the reported torch version against what vLLM's release notes say it requires. Any mismatch here explains most import-time and runtime failures. ### Reproduce the import failure with a full traceback **Command:** python -c "import vllm" **Interpretation:** The traceback usually names the exact missing shared library or incompatible symbol, pointing directly to whether the problem is torch, CUDA, or an attention kernel library. ### Confirm the CUDA toolkit matches what vLLM's build expects **Command:** nvcc --version **Interpretation:** Only relevant if you are building from source; irrelevant for prebuilt wheel installs where torch and vLLM bundle their own CUDA runtime. ### Confirm torch itself works before blaming vLLM **Command:** python -c "import torch; print(torch.__version__, torch.cuda.is_available())" **Interpretation:** If this fails, the problem is your base torch/CUDA/driver setup, not vLLM specifically; fix that layer first. ## Prevention - Always install vLLM into a fresh, dedicated virtual environment rather than an environment with pre-existing ML packages. - Pin the exact vLLM version in requirements.txt or a lockfile so upgrades are deliberate, tested changes. - Check the CUDA and Python compatibility matrix in vLLM's release notes before every upgrade, not just on first install. - Bake a tested vLLM environment into a container image for production so individual machine drift cannot break the install. ## When To Escalate If you are standardizing vLLM across many GPU node types with different drivers and CUDA generations, or need an air-gapped install with no PyPI access, this becomes an image and artifact pipeline problem rather than a one-off pip troubleshooting session. ## Faq ### Should I install torch before or after vLLM? **Answer:** After, or not at all. Let pip install vllm resolve and install its required torch version automatically in a fresh environment. Pre-installing torch yourself is the single most common cause of vLLM's dependency conflict errors, because it locks in a version that may not match what vLLM's kernels were compiled against. ### Can I use vLLM with any CUDA version I want? **Answer:** No. vLLM's prebuilt wheels target specific CUDA major versions, and its custom kernels only work correctly with the CUDA runtime torch itself bundles. Check the release notes for your target vLLM version and match your driver and CUDA choice to what that release documents as supported. ### Why does building vLLM from source take so long and then fail? **Answer:** Source builds compile custom CUDA kernels with nvcc, which is slow and memory-intensive, and depends on having a matching CUDA toolkit and compatible host compiler installed. Most users should use the prebuilt wheel; only build from source for unreleased features and expect to debug compiler-level errors if you do. ### I get a working install on one server but not another. Why? **Answer:** This almost always means the two servers have different driver versions, CUDA toolkits, or pre-existing torch installs. Compare nvidia-smi's CUDA Version and pip show torch output across both machines to find the divergence, then standardize on a container image instead of per-machine pip installs. ## Search terms vllm installation errors, vllm requires torch incompatible, vllm cuda version, pip install vllm failed, vllm import error libcudart, vllm not compatible with cuda version --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json