# Why your vLLM server won't start, and how to diagnose the real cause

> Source: https://netray.co/troubleshoot/vllm-server-fails-to-start
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: On-Prem AI Troubleshooting

## Short answer

vLLM server startup failures collapse into four buckets: the port is already bound by another process, Hugging Face auth is missing or expired for a gated repo, there isn't enough free VRAM for the requested model and context, or the installed vLLM version doesn't yet support the model's architecture. Read the last traceback line, not just the top, to tell them apart.

## Error Title

vLLM server won't start (port in use, auth, VRAM, or unsupported architecture)

## Error Signature

ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use

## Alternate Signatures

- OSError: You are trying to access a gated repo... 401 Client Error
- ValueError: Model architecture 'XxxForCausalLM' is not supported for now

## Category

serving

## Stack

- vllm
- huggingface
- cuda

## Affects

vLLM 0.4 and later, any OS/GPU combination; symptoms differ by root cause but all present as the process exiting before it reaches Uvicorn running

## Quick Fix

### Diagnose in order, fastest first

**Steps:**

- Check the last lines of the traceback for one of: address already in use, 401 Client Error or gated repo, CUDA out of memory, or architecture is not supported.
- Port conflict: pass --port 8001 or kill whatever is already listening on 8000.
- Auth: run huggingface-cli login or export HF_TOKEN, and confirm you accepted the model's license on its Hugging Face page.
- VRAM: lower --max-model-len, add --quantization, or move to a smaller model or more GPUs.
- Unsupported architecture: upgrade vLLM with pip install -U vllm, or check the model card for the minimum vLLM version it requires.

## Symptoms

- Process exits within seconds, never reaching Uvicorn running on http://0.0.0.0:8000.
- Traceback is long and includes framework internals, making it easy to miss the actual root-cause line at the bottom.
- Works for one model but not another on the same machine.
- Worked yesterday, fails today after a vLLM, driver, or model revision change.

## Causes

### Port already bound by a previous vLLM process or another service

**Likelihood:** most-common

**Explanation:** A prior vLLM instance that didn't shut down cleanly, for example killed with SIGKILL instead of SIGTERM, can leave the port held by a zombie process, or another service is already using port 8000. The OS refuses the bind and Python raises an OSError before the app object is even created.

**Fix:** Find and kill the process on that port, or start vLLM on a different --port.

**Commands:**

- lsof -i :8000
- kill -9 PID
- vllm serve MODEL_ID --port 8001

### Gated or private Hugging Face repo without a valid token

**Likelihood:** common

**Explanation:** Meta, Google, and other model owners gate weights behind license acceptance. vLLM's download step hits Hugging Face's API, gets a 401 or 403 because no token is present or the token's account hasn't accepted the license, and the whole process aborts before serving starts.

**Fix:** Log in with huggingface-cli login or export HF_TOKEN, and accept the license on the model page using that same account.

**Commands:**

- huggingface-cli login
- export HF_TOKEN=hf_xxx

### Insufficient VRAM for weights plus KV cache at the requested settings

**Likelihood:** common

**Explanation:** If the model's weights alone don't fit in the visible GPU(s) at the requested dtype, or fit but leave no room for even a minimal KV cache pool, vLLM throws a CUDA OOM during weight loading rather than a graceful message. This is distinct from the KV-cache-sizing ValueError; it happens lower in the startup sequence.

**Fix:** Reduce --max-model-len, add --quantization awq/gptq/fp8, use --tensor-parallel-size across more GPUs, or pick a smaller model.

**Commands:**

- vllm serve MODEL_ID --quantization awq --max-model-len 4096

### Model architecture not yet supported by the installed vLLM version

**Likelihood:** occasional

**Explanation:** New model families ship with novel attention or MoE layouts; vLLM adds support for each architecture in a specific release. Running a brand-new model against an older pinned vLLM raises a clear ValueError naming the unsupported architecture class.

**Fix:** Upgrade vLLM to the version documented on the model card, or wait for a point release if support was only just merged.

**Commands:**

- pip install -U vllm
- python -c "import vllm; print(vllm.__version__)"

### CUDA or driver mismatch with the installed vLLM wheel

**Likelihood:** rare

**Explanation:** vLLM wheels are built against specific CUDA toolkit versions; a driver too old for the compiled CUDA runtime causes low-level initialization failures that surface as import errors or crashes before the HTTP server starts.

**Fix:** Match your vLLM wheel's CUDA version to your driver, or reinstall using the CUDA-version-specific install instructions from vLLM's docs.

**Commands:**

- nvidia-smi
- pip show vllm

## Diagnostics

### Check what's holding the port

**Command:** lsof -i :8000

**Interpretation:** A PID here means something is already listening; kill it or change --port.

### Test Hugging Face auth independently of vLLM

**Command:** python -c "from huggingface_hub import whoami; print(whoami())"

**Interpretation:** An error here confirms it's an auth problem before you even touch vLLM; fix login first.

### Check free VRAM before serving

**Command:** nvidia-smi --query-gpu=memory.used,memory.total --format=csv

**Interpretation:** If used memory is near total before you start, another process or a zombie vLLM instance is consuming the VRAM you need.

### Confirm vLLM version

**Command:** python -c "import vllm; print(vllm.__version__)"

**Interpretation:** Cross-check this version number against the model card's minimum supported vLLM version.

## Prevention

- Use a process manager such as systemd or Kubernetes that ensures old vLLM processes are fully terminated before a new one starts on the same port.
- Bake HF_TOKEN into your deployment secrets rather than relying on an interactive login, especially for CI/CD restarts.
- Pin your vLLM version in requirements and test new model releases in staging before assuming compatibility.
- Add a startup health check that fails fast and loud in your orchestrator, rather than silently retrying a broken config.

## When To Escalate

If every model you try fails to load on VRAM grounds, or you're constantly chasing vLLM version bumps to support new architectures, that's a sign your hardware or your model-release process needs redesigning, not another flag tweak.

## Faq

### Why does the traceback look so long when the actual problem is simple?

**Answer:** vLLM wraps Ray, PyTorch, and Hugging Face internals, so a single root cause like a 401 can print through several layers of framework code. Always read the last few lines of the traceback, where the actual exception type and message live, rather than the top.

### Do I need a token for every model?

**Answer:** No, only for gated or private repos, which includes many Meta and Google releases requiring license acceptance. Fully open weights like most Qwen, DeepSeek, and Mistral community releases don't need a token.

### Is it safe to just keep retrying with pip install -U vllm?

**Answer:** It's a reasonable first step for unsupported architecture errors, but pin the resulting version afterward and test it against your other models before rolling to production, since a vLLM upgrade can also change default behavior for models you already run.

## Search terms

vllm server fails to start, address already in use vllm, vllm 401 gated repo, vllm architecture not supported, vllm won't start, vllm startup error, vllm port in use

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json