# Why vLLM's OpenAI-compatible API returns 404, and how to fix the request

> Source: https://netray.co/troubleshoot/openai-compatible-endpoint-404
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: On-Prem AI Troubleshooting

## Short answer

This 404 is almost always a client-side mismatch, not a server bug: either the request hit the wrong route, such as missing the /v1 prefix, or the model field in the request body doesn't match the exact served-model-name (or default model repo id) vLLM registered at startup. Fix the URL and model name to match what /v1/models actually reports.

## Error Title

vLLM OpenAI-compatible API returns 404 Not Found

## Error Signature

{"object":"error","message":"The model `gpt-3.5-turbo` does not exist.","type":"NotFoundError","code":404}

## Alternate Signatures

- 404 Not Found calling /chat/completions (missing the /v1 prefix)

## Category

serving

## Stack

- vllm

## Affects

vLLM 0.3 and later, any deployment; also applies to other OpenAI-compatible servers with equivalent model-name or route mismatches

## Quick Fix

### Fix it in under a minute

**Steps:**

- Call GET /v1/models against your server and copy the exact id value it returns.
- Use that exact string as the model field in your request body, character for character.
- Confirm your request URL includes the /v1 prefix, for example http://host:8000/v1/chat/completions, not just /chat/completions.
- If you set --served-model-name at launch, that alias, not the Hugging Face repo id, is what clients must send.
- Re-test with curl before touching client SDK code, to isolate whether the problem is the server or the client library.

## Symptoms

- Server is up and /v1/models returns a valid list, but chat completions requests 404.
- Works with curl using the exact model id but fails from an application using a hardcoded model name like gpt-3.5-turbo or gpt-4.
- Switching between --served-model-name and the raw Hugging Face repo id changes whether requests succeed.
- Client library defaults to a path that omits the /v1 prefix vLLM expects.

## Causes

### Request's model field doesn't match the server's registered model name

**Likelihood:** most-common

**Explanation:** Many OpenAI-SDK-based clients default to a model string like gpt-3.5-turbo, hardcoded for a previous provider. vLLM only recognizes the exact Hugging Face repo id it loaded, or the alias given with served-model-name, and returns 404 for anything else because from its perspective that model simply isn't loaded.

**Fix:** Set the client's model parameter to exactly match the id returned by GET /v1/models, or launch vLLM with --served-model-name set to the friendly name your client already expects.

**Commands:**

- curl http://localhost:8000/v1/models
- vllm serve MODEL_ID --served-model-name gpt-3.5-turbo

### Missing /v1 prefix in the request path

**Likelihood:** common

**Explanation:** vLLM's OpenAI-compatible server namespaces every endpoint under /v1, mirroring OpenAI's own API. A request to /chat/completions instead of /v1/chat/completions hits no registered route at all, and the ASGI framework returns a generic 404, not a vLLM-specific error.

**Fix:** Point your base_url, or the OpenAI client's base URL config, at http://host:port/v1, not just http://host:port.

**Commands:**

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d "{\"model\":\"MODEL_ID\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}"

### Hitting an endpoint vLLM doesn't implement for the version you're running

**Likelihood:** common

**Explanation:** Not every OpenAI API surface is implemented in every vLLM release. Calling an endpoint like /v1/embeddings or an audio endpoint on a version or model that doesn't support it returns 404 even though other endpoints work fine on the same server.

**Fix:** Check the vLLM release notes and docs for your installed version to confirm the endpoint and model type you're calling are actually supported.

### Load balancer or reverse proxy stripping or rewriting the /v1 path

**Likelihood:** occasional

**Explanation:** An nginx, API gateway, or ingress rule that rewrites paths, common when proxying multiple backend services under one domain, can accidentally strip the /v1 segment before the request reaches vLLM, producing a 404 that looks like a client bug but is actually a proxy misconfiguration.

**Fix:** Curl the vLLM server directly, bypassing the proxy, to confirm the same request succeeds, then fix the proxy's path rewrite rule.

**Commands:**

- curl http://vllm-pod-ip:8000/v1/models

### Client SDK version expects a route vLLM doesn't expose

**Likelihood:** rare

**Explanation:** Newer OpenAI SDK versions add new endpoints, such as a responses API, that vLLM's OpenAI-compatibility layer may not implement yet, so calls using the newest SDK conventions 404 even though older chat completions calls work fine.

**Fix:** Use the chat completions or completions client methods explicitly rather than a newer SDK convenience method vLLM doesn't yet mirror.

## Diagnostics

### List models the server actually has loaded

**Command:** curl http://localhost:8000/v1/models

**Interpretation:** The id field(s) returned are the only valid values for the model parameter in requests; anything else 404s.

### Test the canonical chat completions route directly with curl

**Command:** curl -s http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d "{\"model\":\"MODEL_ID\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}"

**Interpretation:** If this succeeds but your app still 404s, the bug is in the client's base URL or model name, not the server.

### Confirm no proxy is rewriting the path

**Command:** curl http://VLLM_INTERNAL_IP:8000/v1/models

**Interpretation:** Comparing the direct-to-pod result against the result through your load balancer or ingress isolates whether a reverse proxy is the culprit.

## Prevention

- Standardize on setting --served-model-name explicitly at launch so client code never has to track raw Hugging Face repo ids.
- Add an automated smoke test that hits /v1/models and /v1/chat/completions after every deployment before routing real traffic.
- Document the exact base URL, including /v1, and model name in onboarding docs for any team consuming the endpoint.
- Version-pin your OpenAI client SDK and test against vLLM's supported endpoint surface before upgrading either side.

## When To Escalate

If you're running many models behind one gateway and constantly hitting name-mismatch 404s across teams, that's a routing and naming-convention problem worth solving with a model registry or API gateway layer rather than repeatedly fixing individual client configs.

## Faq

### Why does vLLM care about the exact model name if it only has one model loaded?

**Answer:** It mirrors OpenAI's multi-model API contract on purpose, so existing OpenAI-client code, including load balancers routing by model name, works unmodified. That fidelity means it validates the model field strictly rather than ignoring it, which is why an unset or wrong default 404s.

### Can I make vLLM accept any model name so my app doesn't need changes?

**Answer:** Yes, launch it with --served-model-name set to whatever string your application already sends, such as gpt-3.5-turbo, and vLLM will register that as the accepted model id even though it's serving a completely different underlying model.

### Is a 404 always the client's fault?

**Answer:** Usually yes for this specific error, but always confirm by curling the server directly first. If a reverse proxy or gateway sits in front of vLLM, path rewriting there can produce an identical-looking 404 that has nothing to do with your application code.

## Search terms

vllm 404 not found, openai compatible api 404, vllm model does not exist error, served-model-name vllm, vllm v1 chat completions 404, vllm api endpoint not found

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json