# Why vLLM's multi-LoRA serving fails, and how to fix rank and capacity mismatches > Source: https://netray.co/troubleshoot/vllm-multi-lora-serving-errors > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: On-Prem AI Troubleshooting ## Short answer Multi-LoRA serving in vLLM requires the server to be launched with --enable-lora plus explicit capacity flags such as max-lora-rank, max-loras, and max-cpu-loras sized to your actual adapters, and each adapter must be registered by name with --lora-modules so requests can reference it. Mismatches between an adapter's real rank or count and what the server was configured for produce hard failures rather than silent truncation. ## Error Title vLLM multi-LoRA serving fails to load or apply an adapter ## Error Signature ValueError: LoRA rank X is greater than max_lora_rank Y ## Alternate Signatures - ValueError: LoRA adapter 'name' not found, did you forget to register it with --lora-modules? - RuntimeError: Number of LoRAs (X) exceeds max_loras (Y) ## Category serving ## Stack - vllm - huggingface ## Affects vLLM 0.4 and later with --enable-lora; occurs when adapter rank, adapter count, or naming doesn't match how the server was launched ## Quick Fix ### Get multi-LoRA serving working **Steps:** - Launch vLLM with --enable-lora and register every adapter explicitly, for example --lora-modules name1=/path/to/adapter1 name2=/path/to/adapter2. - Check each adapter's adapter_config.json for its rank value and set --max-lora-rank to at least the largest rank among your adapters. - Set --max-loras to at least the number of adapters you need active concurrently on the GPU at once. - In requests, set the model field to the exact adapter name you registered, not the base model name, to route to that LoRA. - If you need more adapters resident than GPU memory allows concurrently, raise --max-cpu-loras to let vLLM swap adapters from CPU RAM. ## Symptoms - Server starts fine for the base model but errors as soon as a request references a specific LoRA adapter name. - Error explicitly names a rank or count mismatch, such as LoRA rank exceeding max_lora_rank, or number of LoRAs exceeding max_loras. - Adding a new adapter to an already-running deployment fails even though existing adapters work. - Request succeeds when model is the base model id but fails when set to an adapter's registered name. ## Causes ### Adapter's rank exceeds the server's configured max_lora_rank **Likelihood:** most-common **Explanation:** vLLM preallocates fixed-size buffers for LoRA computation sized by max_lora_rank at startup, as part of the same paged-memory design used for the KV cache. An adapter trained at a higher rank than that ceiling literally cannot fit in the preallocated buffers, so vLLM rejects it rather than truncating the adapter's weights. **Fix:** Check the adapter's adapter_config.json for its rank value and relaunch the server with --max-lora-rank set to at least the highest rank among all adapters you plan to serve. **Commands:** - grep rank adapter_config.json - vllm serve BASE_MODEL_ID --enable-lora --max-lora-rank 64 ### Adapter not registered via --lora-modules at launch **Likelihood:** common **Explanation:** vLLM only knows about LoRA adapters explicitly named and pointed to a path at server startup, or added afterward through a dedicated load-adapter endpoint in newer versions. Referencing an adapter name in a request that was never registered produces a not-found error indistinguishable at first glance from a base-model routing problem. **Fix:** Add every adapter you intend to serve to --lora-modules at launch, or use the dynamic adapter-loading endpoint if your vLLM version supports it, before referencing it by name in requests. **Commands:** - vllm serve BASE_MODEL_ID --enable-lora --lora-modules support-bot=/models/adapters/support-bot ### Number of concurrently requested adapters exceeds max_loras **Likelihood:** common **Explanation:** max_loras caps how many distinct LoRA adapters vLLM keeps resident on GPU at once for concurrent serving. Requesting more distinct adapters simultaneously than this ceiling forces vLLM to either queue, evict, or reject requests depending on version and configuration, surfacing as a capacity error under real concurrent multi-tenant traffic. **Fix:** Raise --max-loras to match your expected concurrent adapter count, and use --max-cpu-loras to allow additional adapters to be swapped in from CPU RAM rather than rejected. **Commands:** - vllm serve BASE_MODEL_ID --enable-lora --max-loras 8 --max-cpu-loras 16 ### Base model and adapter architecture mismatch **Likelihood:** occasional **Explanation:** A LoRA adapter is trained against a specific base model's exact layer shapes and target modules. Attempting to apply an adapter trained for a different base model, even a closely related same-family variant, produces shape mismatches during the low-rank matrix multiplication, since the adapter's dimensions don't align with the serving model's weight matrices. **Fix:** Confirm the adapter's base_model_name_or_path in adapter_config.json exactly matches the model you're serving it against. **Commands:** - grep base_model_name_or_path adapter_config.json ### enable-lora omitted entirely, so LoRA-related flags are silently ignored **Likelihood:** rare **Explanation:** All LoRA capacity and registration flags only take effect if --enable-lora is also passed. Without it, vLLM runs in plain base-model mode and any --lora-modules or --max-lora-rank flags on the same command line have no effect, making adapter requests fail as if no LoRA support exists at all. **Fix:** Double check --enable-lora is present in the exact launch command being used, not just the LoRA-specific sizing flags. **Commands:** - vllm serve BASE_MODEL_ID --enable-lora --lora-modules name=/path ## Diagnostics ### Check an adapter's actual rank and target base model **Command:** cat /path/to/adapter/adapter_config.json **Interpretation:** Confirm the rank value is at or below your server's max_lora_rank, and base_model_name_or_path matches the model you're serving against. ### Confirm the server was launched with LoRA support and expected registrations **Command:** ps aux | grep vllm **Interpretation:** Missing --enable-lora means no LoRA flags take effect at all, regardless of what else was passed. ### List models and adapters the running server recognizes **Command:** curl http://localhost:8000/v1/models **Interpretation:** Each registered adapter should appear as its own entry alongside the base model; if your adapter name is missing, it was never registered at launch. ## Prevention - Standardize a pre-deployment check that reads every adapter's adapter_config.json rank and validates it against the fleet's max-lora-rank before rollout. - Track adapter-to-base-model compatibility explicitly in your model registry so a mismatched pairing is caught before it reaches the server. - Size max-loras and max-cpu-loras deliberately based on real concurrent-tenant projections, not just the number of adapters that exist today. - Add an automated test that round-trips a request through every registered adapter name after each deployment, not just the base model. ## When To Escalate If you're serving dozens of adapters with highly varied ranks and constantly hitting capacity ceilings, that's a multi-tenant serving architecture question, such as dedicated adapter-serving pools, rank standardization across fine-tuning jobs, or a separate LoRA-serving tier, worth designing deliberately rather than continuously raising max-loras. ## Faq ### Why does vLLM need to know the max LoRA rank in advance? **Answer:** It preallocates fixed-size GPU buffers for the LoRA matrix multiplications at startup, the same paged-memory design it uses for the KV cache, so every adapter's rank must fit within a ceiling declared before any requests arrive, not discovered dynamically per request. ### Can I serve adapters trained for a different but similar base model? **Answer:** No, a LoRA adapter's low-rank matrices are shaped specifically for its exact training base model's weight dimensions. Using it against a different model, even a closely related one, causes shape mismatches rather than degraded-but-working output. ### What's the difference between max_loras and max_cpu_loras? **Answer:** max_loras caps how many adapters can be resident on GPU simultaneously for active serving. max_cpu_loras lets additional adapters beyond that GPU-resident count sit in CPU RAM and get swapped in on demand, trading some latency for supporting a larger total adapter catalog. ## Search terms vllm multi-lora error, vllm lora rank error, max_lora_rank vllm, vllm lora-modules not found, vllm enable-lora, vllm serving multiple adapters --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json