# Why your LoRA adapter won't load onto the base model, and how to fix it > Source: https://netray.co/troubleshoot/lora-adapter-not-loading > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: On-Prem AI Troubleshooting ## Short answer A LoRA adapter usually fails to load because it was trained against a different base model than the one you're loading it onto, because the rank or alpha in adapter_config.json doesn't match what was actually trained, because the installed PEFT version is incompatible with how the adapter was saved, or because the adapter directory is missing its config file. Check adapter_config.json's base_model_name_or_path first. ## Error Title LoRA adapter fails to load onto the base model ## Error Signature ValueError: Target modules {'q_proj', 'v_proj'} not found in the base model ## Alternate Signatures - RuntimeError: Error(s) in loading state_dict for PeftModel: size mismatch - Some weights of the model checkpoint were not used when initializing the adapter ## Category fine-tuning ## Stack - huggingface - transformers - pytorch ## Affects PEFT-based LoRA and QLoRA adapters loaded with transformers and PEFT, any base model family ## Quick Fix ### Fastest path to a loading adapter **Steps:** Open adapter_config.json in the adapter directory and confirm base_model_name_or_path exactly matches the base model you are loading (including quantization variant). Confirm both adapter_model.safetensors (or .bin) and adapter_config.json exist in the same directory; a missing config file is a common cause of a silent or confusing failure. Check that your installed peft and transformers versions are equal to or newer than the versions used to train and save the adapter (pip show peft transformers). Load with PeftModel.from_pretrained(base_model, adapter_path, is_trainable=False) rather than merging manually, and read the full traceback for which specific keys mismatched. If rank/alpha differ, re-check the LoraConfig used at training time (saved in adapter_config.json) against what your loading code assumes, and load using the config's own values instead of hardcoding them. ## Symptoms - Loading raises a size mismatch error naming specific lora_A or lora_B weight tensors - Loading raises a target modules not found error listing module names that don't exist on this base model - Adapter loads without error but generations are identical to the unmodified base model - PeftModel.from_pretrained raises a missing key or unexpected key error during state_dict loading - Adapter directory only contains a .bin/.safetensors file with no adapter_config.json ## Causes ### Adapter trained on a different base model than the one being loaded **Likelihood:** most-common **Explanation:** LoRA weights are low-rank deltas shaped to fit specific layer dimensions and module names of the exact base model architecture they were trained against. Loading onto a different model (even a same-size variant with a different tokenizer or a differently quantized version) produces shape mismatches or references to module names that don't exist in the new model's structure. **Fix:** Read base_model_name_or_path from adapter_config.json and load that exact base model and revision. If you intentionally want to use a different base, you must retrain the adapter against it. **Commands:** - python -c "import json; print(json.load(open('adapter_config.json'))['base_model_name_or_path'])" ### Rank or alpha in config drifted from what was actually trained **Likelihood:** common **Explanation:** If adapter_config.json was hand-edited, regenerated, or copied from a different run after training, the r and lora_alpha values it reports may no longer match the actual shapes of the saved lora_A/lora_B weight tensors, causing PEFT to build layers of the wrong size before it tries to load the real weights into them. **Fix:** Regenerate adapter_config.json from the same training run's TrainingArguments/LoraConfig, or inspect the actual tensor shapes in adapter_model.safetensors and correct r to match rather than trusting a possibly stale config file. ### Incompatible PEFT version between training and loading environments **Likelihood:** common **Explanation:** PEFT has changed its internal state_dict key naming and target module resolution logic across versions. An adapter saved with an older or newer PEFT version can use key names or a config schema that a mismatched version does not recognize, producing missing key or unexpected key errors even though the adapter itself is otherwise valid. **Fix:** Pin the exact peft version used for training in your inference environment's requirements, or upgrade both environments to the same recent version and re-save the adapter if needed. **Commands:** - pip show peft - pip install peft== ### Adapter saved without its config, only the weight file **Likelihood:** occasional **Explanation:** Some custom training scripts save only model.state_dict() output or only the raw adapter weight tensors, skipping adapter_config.json entirely. Without the config, PEFT has no information about target_modules, rank, or alpha, and cannot reconstruct the adapter architecture needed to load the weights correctly. **Fix:** Always save adapters with model.save_pretrained(adapter_dir) from a PeftModel object, which writes both the weights and adapter_config.json together. If only weights exist, manually recreate the config from your training script's LoraConfig and place it alongside the weights. ### Attempting to load a merged model's adapter as if it were unmerged **Likelihood:** rare **Explanation:** If the adapter was merged into the base model with merge_and_unload() and then saved as a full model, there is no separate adapter to load; the resulting directory is a complete standalone model, not a PEFT adapter. Calling PeftModel.from_pretrained on it fails because there is no adapter_config.json or adapter weight file. **Fix:** Load merged models directly with AutoModelForCausalLM.from_pretrained rather than through PeftModel; only use the PEFT loading path for directories that actually contain an unmerged adapter. ## Diagnostics ### Inspect adapter config contents **Command:** cat adapter_config.json **Interpretation:** Confirm base_model_name_or_path, r, lora_alpha, and target_modules all match what you expect from the training run. Any mismatch here explains most loading failures before you even attempt to load. ### List adapter directory contents **Command:** ls -la adapter_path/ **Interpretation:** You should see both adapter_config.json and adapter_model.safetensors (or adapter_model.bin). If either is missing, the adapter was saved incompletely and needs to be re-exported from the training checkpoint. ### Check installed PEFT and transformers versions **Command:** pip show peft transformers **Interpretation:** Compare against the versions recorded (if logged) at training time. A large version gap, especially a major version bump in peft, is a common source of key mismatch errors during load. ### Inspect actual tensor shapes in the adapter weights **Command:** python -c "from safetensors import safe_open; f=safe_open('adapter_model.safetensors','pt'); [print(k, f.get_tensor(k).shape) for k in list(f.keys())[:6]]" **Interpretation:** The rank dimension visible in lora_A and lora_B tensor shapes should match the r value in adapter_config.json. A mismatch confirms the config file drifted from the actual trained weights. ## Prevention - Always save adapters via the PeftModel's own save_pretrained method so config and weights stay bundled together. - Record the exact base model revision (commit hash, not just the repo name) alongside every saved adapter. - Pin peft and transformers versions in a requirements/lockfile shared between training and serving environments. - Add an automated load-test step in CI that loads every newly trained adapter onto its intended base model before deployment. - Store adapter metadata (rank, alpha, target_modules, training data version) in a small manifest file next to the adapter for future audits. ## When To Escalate If you are managing many adapters across multiple base model versions and rank/config drift keeps recurring despite process fixes, that points to a need for a proper adapter registry and versioning system rather than manual file management. If adapters need to be swapped in and out of a production serving layer (like vLLM's multi-LoRA support) reliably, that deployment architecture is worth designing deliberately rather than debugging one incident at a time. ## Faq ### Why does my LoRA adapter say target modules not found? **Answer:** This means adapter_config.json lists module names (like q_proj or v_proj) that don't exist under those names in the base model you're loading. This happens when the base model architecture differs from the one used at training time, even if the model sizes and names look similar. ### Can I load a LoRA adapter trained with an older PEFT version? **Answer:** Usually yes, but not always. PEFT maintains reasonable backward compatibility, but major version jumps have changed state_dict key naming in the past. If loading fails with missing or unexpected key errors, try installing the exact PEFT version used during training first to confirm compatibility. ### What files do I need to load a LoRA adapter? **Answer:** You need both adapter_config.json (describing rank, alpha, and target modules) and the weight file, typically adapter_model.safetensors or adapter_model.bin. Both should be produced together by calling save_pretrained on a PeftModel; having only one of the two will cause loading to fail. ### Can I merge a LoRA adapter into a different base model than it was trained on? **Answer:** No. The low-rank weight deltas are mathematically tied to the exact parameter shapes and representations of the base model used during training. Merging onto a different model produces nonsensical outputs even if the merge operation itself does not error out. ## Search terms LoRA adapter not loading, target modules not found peft, adapter_config.json mismatch, peft version compatibility, size mismatch loading state_dict, base_model_name_or_path error, LoRA rank alpha mismatch, PeftModel from_pretrained error --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json