Why vLLM's multi-LoRA serving fails, and how to fix rank and capacity mismatches
ValueError: LoRA rank X is greater than max_lora_rank Y
Also appears as
- ValueError: LoRA adapter 'name' not found, did you forget to register it with --lora-modules?
- RuntimeError: Number of LoRAs (X) exceeds max_loras (Y)
Short answer
Multi-LoRA serving in vLLM requires the server to be launched with --enable-lora plus explicit capacity flags such as max-lora-rank, max-loras, and max-cpu-loras sized to your actual adapters, and each adapter must be registered by name with --lora-modules so requests can reference it. Mismatches between an adapter's real rank or count and what the server was configured for produce hard failures rather than silent truncation.
Affects: vLLM 0.4 and later with --enable-lora; occurs when adapter rank, adapter count, or naming doesn't match how the server was launched
Get multi-LoRA serving working
- 1Launch vLLM with --enable-lora and register every adapter explicitly, for example --lora-modules name1=/path/to/adapter1 name2=/path/to/adapter2.
- 2Check each adapter's adapter_config.json for its rank value and set --max-lora-rank to at least the largest rank among your adapters.
- 3Set --max-loras to at least the number of adapters you need active concurrently on the GPU at once.
- 4In requests, set the model field to the exact adapter name you registered, not the base model name, to route to that LoRA.
- 5If you need more adapters resident than GPU memory allows concurrently, raise --max-cpu-loras to let vLLM swap adapters from CPU RAM.
How to confirm this is your problem
- Server starts fine for the base model but errors as soon as a request references a specific LoRA adapter name.
- Error explicitly names a rank or count mismatch, such as LoRA rank exceeding max_lora_rank, or number of LoRAs exceeding max_loras.
- Adding a new adapter to an already-running deployment fails even though existing adapters work.
- Request succeeds when model is the base model id but fails when set to an adapter's registered name.
Root causes and fixes
Adapter's rank exceeds the server's configured max_lora_rank
vLLM preallocates fixed-size buffers for LoRA computation sized by max_lora_rank at startup, as part of the same paged-memory design used for the KV cache. An adapter trained at a higher rank than that ceiling literally cannot fit in the preallocated buffers, so vLLM rejects it rather than truncating the adapter's weights.
Fix: Check the adapter's adapter_config.json for its rank value and relaunch the server with --max-lora-rank set to at least the highest rank among all adapters you plan to serve.
grep rank adapter_config.json vllm serve BASE_MODEL_ID --enable-lora --max-lora-rank 64
Adapter not registered via --lora-modules at launch
vLLM only knows about LoRA adapters explicitly named and pointed to a path at server startup, or added afterward through a dedicated load-adapter endpoint in newer versions. Referencing an adapter name in a request that was never registered produces a not-found error indistinguishable at first glance from a base-model routing problem.
Fix: Add every adapter you intend to serve to --lora-modules at launch, or use the dynamic adapter-loading endpoint if your vLLM version supports it, before referencing it by name in requests.
vllm serve BASE_MODEL_ID --enable-lora --lora-modules support-bot=/models/adapters/support-bot
Number of concurrently requested adapters exceeds max_loras
max_loras caps how many distinct LoRA adapters vLLM keeps resident on GPU at once for concurrent serving. Requesting more distinct adapters simultaneously than this ceiling forces vLLM to either queue, evict, or reject requests depending on version and configuration, surfacing as a capacity error under real concurrent multi-tenant traffic.
Fix: Raise --max-loras to match your expected concurrent adapter count, and use --max-cpu-loras to allow additional adapters to be swapped in from CPU RAM rather than rejected.
vllm serve BASE_MODEL_ID --enable-lora --max-loras 8 --max-cpu-loras 16
Base model and adapter architecture mismatch
A LoRA adapter is trained against a specific base model's exact layer shapes and target modules. Attempting to apply an adapter trained for a different base model, even a closely related same-family variant, produces shape mismatches during the low-rank matrix multiplication, since the adapter's dimensions don't align with the serving model's weight matrices.
Fix: Confirm the adapter's base_model_name_or_path in adapter_config.json exactly matches the model you're serving it against.
grep base_model_name_or_path adapter_config.json
enable-lora omitted entirely, so LoRA-related flags are silently ignored
All LoRA capacity and registration flags only take effect if --enable-lora is also passed. Without it, vLLM runs in plain base-model mode and any --lora-modules or --max-lora-rank flags on the same command line have no effect, making adapter requests fail as if no LoRA support exists at all.
Fix: Double check --enable-lora is present in the exact launch command being used, not just the LoRA-specific sizing flags.
vllm serve BASE_MODEL_ID --enable-lora --lora-modules name=/path
Diagnostic commands
Check an adapter's actual rank and target base model
cat /path/to/adapter/adapter_config.json
Confirm the rank value is at or below your server's max_lora_rank, and base_model_name_or_path matches the model you're serving against.
Confirm the server was launched with LoRA support and expected registrations
ps aux | grep vllm
Missing --enable-lora means no LoRA flags take effect at all, regardless of what else was passed.
List models and adapters the running server recognizes
curl http://localhost:8000/v1/models
Each registered adapter should appear as its own entry alongside the base model; if your adapter name is missing, it was never registered at launch.
Stopping it from happening again
- Standardize a pre-deployment check that reads every adapter's adapter_config.json rank and validates it against the fleet's max-lora-rank before rollout.
- Track adapter-to-base-model compatibility explicitly in your model registry so a mismatched pairing is caught before it reaches the server.
- Size max-loras and max-cpu-loras deliberately based on real concurrent-tenant projections, not just the number of adapters that exist today.
- Add an automated test that round-trips a request through every registered adapter name after each deployment, not just the base model.
When this becomes an architecture problem
If you're serving dozens of adapters with highly varied ranks and constantly hitting capacity ceilings, that's a multi-tenant serving architecture question, such as dedicated adapter-serving pools, rank standardization across fine-tuning jobs, or a separate LoRA-serving tier, worth designing deliberately rather than continuously raising max-loras.
Frequently asked questions
Why does vLLM need to know the max LoRA rank in advance?
It preallocates fixed-size GPU buffers for the LoRA matrix multiplications at startup, the same paged-memory design it uses for the KV cache, so every adapter's rank must fit within a ceiling declared before any requests arrive, not discovered dynamically per request.
Can I serve adapters trained for a different but similar base model?
No, a LoRA adapter's low-rank matrices are shaped specifically for its exact training base model's weight dimensions. Using it against a different model, even a closely related one, causes shape mismatches rather than degraded-but-working output.
What's the difference between max_loras and max_cpu_loras?
max_loras caps how many adapters can be resident on GPU simultaneously for active serving. max_cpu_loras lets additional adapters beyond that GPU-resident count sit in CPU RAM and get swapped in on demand, trading some latency for supporting a larger total adapter catalog.
Size it properly next time
Free calculators that prevent this class of failure before you provision hardware.
LoRA Fine-Tuning Cost Calculator
Turn model size, dataset tokens, epochs, and rank into a GPU-hour and dollar estimate for a LoRA fine-tuning run on rented or owned hardware.
Free ToolQLoRA vs Full Fine-Tuning Cost Calculator
See the GPU memory footprint, GPU-hour requirement, and dollar cost gap between QLoRA and full fine-tuning for the same model size and dataset.
Free ToolGPU Sizing Calculator for LLM Inference
Work out how many GPUs you need to serve a given open-weight model to your user base, based on memory footprint and token throughput.
Related problems
LoRA adapter fails to load onto the base model
A LoRA adapter usually fails to load because it was trained against a different base model than the one you're loading it onto, because the rank or alpha in adapter_config.json doesn't match what was actually trained, because the installed PEFT version is incompatible with how the adapter was saved, or because the adapter directory is missing its config file. Check adapter_config.json's base_model_name_or_path first.
vLLM runs out of memory during startup, before serving any requests
vLLM's startup OOMs happen because it preallocates a KV cache pool sized against gpu_memory_utilization right after loading weights, so the failure point is engine initialization, not user traffic. Fix it by lowering gpu_memory_utilization if it's set too aggressively for actual free VRAM, lowering max_model_len, or reducing weight footprint with quantization or more GPUs.
vLLM server won't start (port in use, auth, VRAM, or unsupported architecture)
vLLM server startup failures collapse into four buckets: the port is already bound by another process, Hugging Face auth is missing or expired for a gated repo, there isn't enough free VRAM for the requested model and context, or the installed vLLM version doesn't yet support the model's architecture. Read the last traceback line, not just the top, to tell them apart.
GuideLoRA vs QLoRA: Choosing the Right Fine-Tuning Method
LoRA vs QLoRA for enterprise fine-tuning: rank and alpha choices, real VRAM math by model size, and when each method actually wins.
GuidevLLM Production Deployment: A Practitioner's Guide
Deploy vLLM in production: continuous batching, PagedAttention, config flags that matter, and the metrics to watch before you trust it with real traffic.
GuideFine-Tuning LLMs On-Prem with Enterprise Data
Fine-tune LLMs on-prem with enterprise data: LoRA vs full fine-tuning, dataset prep, GPU requirements, eval, and when RAG beats tuning altogether.
Still stuck, or tired of fighting your own infrastructure?
Netray deploys and operates on-prem AI for regulated manufacturers and defense suppliers. We have debugged this stack in production, on air-gapped networks, at scale.