Why your LoRA adapter won't load onto the base model, and how to fix it
ValueError: Target modules {'q_proj', 'v_proj'} not found in the base modelAlso appears as
- RuntimeError: Error(s) in loading state_dict for PeftModel: size mismatch
- Some weights of the model checkpoint were not used when initializing the adapter
Short answer
A LoRA adapter usually fails to load because it was trained against a different base model than the one you're loading it onto, because the rank or alpha in adapter_config.json doesn't match what was actually trained, because the installed PEFT version is incompatible with how the adapter was saved, or because the adapter directory is missing its config file. Check adapter_config.json's base_model_name_or_path first.
Affects: PEFT-based LoRA and QLoRA adapters loaded with transformers and PEFT, any base model family
Fastest path to a loading adapter
- 1Open adapter_config.json in the adapter directory and confirm base_model_name_or_path exactly matches the base model you are loading (including quantization variant).
- 2Confirm both adapter_model.safetensors (or .bin) and adapter_config.json exist in the same directory; a missing config file is a common cause of a silent or confusing failure.
- 3Check that your installed peft and transformers versions are equal to or newer than the versions used to train and save the adapter (pip show peft transformers).
- 4Load with PeftModel.from_pretrained(base_model, adapter_path, is_trainable=False) rather than merging manually, and read the full traceback for which specific keys mismatched.
- 5If rank/alpha differ, re-check the LoraConfig used at training time (saved in adapter_config.json) against what your loading code assumes, and load using the config's own values instead of hardcoding them.
How to confirm this is your problem
- Loading raises a size mismatch error naming specific lora_A or lora_B weight tensors
- Loading raises a target modules not found error listing module names that don't exist on this base model
- Adapter loads without error but generations are identical to the unmodified base model
- PeftModel.from_pretrained raises a missing key or unexpected key error during state_dict loading
- Adapter directory only contains a .bin/.safetensors file with no adapter_config.json
Root causes and fixes
Adapter trained on a different base model than the one being loaded
LoRA weights are low-rank deltas shaped to fit specific layer dimensions and module names of the exact base model architecture they were trained against. Loading onto a different model (even a same-size variant with a different tokenizer or a differently quantized version) produces shape mismatches or references to module names that don't exist in the new model's structure.
Fix: Read base_model_name_or_path from adapter_config.json and load that exact base model and revision. If you intentionally want to use a different base, you must retrain the adapter against it.
python -c "import json; print(json.load(open('adapter_config.json'))['base_model_name_or_path'])"Rank or alpha in config drifted from what was actually trained
If adapter_config.json was hand-edited, regenerated, or copied from a different run after training, the r and lora_alpha values it reports may no longer match the actual shapes of the saved lora_A/lora_B weight tensors, causing PEFT to build layers of the wrong size before it tries to load the real weights into them.
Fix: Regenerate adapter_config.json from the same training run's TrainingArguments/LoraConfig, or inspect the actual tensor shapes in adapter_model.safetensors and correct r to match rather than trusting a possibly stale config file.
Incompatible PEFT version between training and loading environments
PEFT has changed its internal state_dict key naming and target module resolution logic across versions. An adapter saved with an older or newer PEFT version can use key names or a config schema that a mismatched version does not recognize, producing missing key or unexpected key errors even though the adapter itself is otherwise valid.
Fix: Pin the exact peft version used for training in your inference environment's requirements, or upgrade both environments to the same recent version and re-save the adapter if needed.
pip show peft pip install peft==<training-version>
Adapter saved without its config, only the weight file
Some custom training scripts save only model.state_dict() output or only the raw adapter weight tensors, skipping adapter_config.json entirely. Without the config, PEFT has no information about target_modules, rank, or alpha, and cannot reconstruct the adapter architecture needed to load the weights correctly.
Fix: Always save adapters with model.save_pretrained(adapter_dir) from a PeftModel object, which writes both the weights and adapter_config.json together. If only weights exist, manually recreate the config from your training script's LoraConfig and place it alongside the weights.
Attempting to load a merged model's adapter as if it were unmerged
If the adapter was merged into the base model with merge_and_unload() and then saved as a full model, there is no separate adapter to load; the resulting directory is a complete standalone model, not a PEFT adapter. Calling PeftModel.from_pretrained on it fails because there is no adapter_config.json or adapter weight file.
Fix: Load merged models directly with AutoModelForCausalLM.from_pretrained rather than through PeftModel; only use the PEFT loading path for directories that actually contain an unmerged adapter.
Diagnostic commands
Inspect adapter config contents
cat adapter_config.json
Confirm base_model_name_or_path, r, lora_alpha, and target_modules all match what you expect from the training run. Any mismatch here explains most loading failures before you even attempt to load.
List adapter directory contents
ls -la adapter_path/
You should see both adapter_config.json and adapter_model.safetensors (or adapter_model.bin). If either is missing, the adapter was saved incompletely and needs to be re-exported from the training checkpoint.
Check installed PEFT and transformers versions
pip show peft transformers
Compare against the versions recorded (if logged) at training time. A large version gap, especially a major version bump in peft, is a common source of key mismatch errors during load.
Inspect actual tensor shapes in the adapter weights
python -c "from safetensors import safe_open; f=safe_open('adapter_model.safetensors','pt'); [print(k, f.get_tensor(k).shape) for k in list(f.keys())[:6]]"The rank dimension visible in lora_A and lora_B tensor shapes should match the r value in adapter_config.json. A mismatch confirms the config file drifted from the actual trained weights.
Stopping it from happening again
- Always save adapters via the PeftModel's own save_pretrained method so config and weights stay bundled together.
- Record the exact base model revision (commit hash, not just the repo name) alongside every saved adapter.
- Pin peft and transformers versions in a requirements/lockfile shared between training and serving environments.
- Add an automated load-test step in CI that loads every newly trained adapter onto its intended base model before deployment.
- Store adapter metadata (rank, alpha, target_modules, training data version) in a small manifest file next to the adapter for future audits.
When this becomes an architecture problem
If you are managing many adapters across multiple base model versions and rank/config drift keeps recurring despite process fixes, that points to a need for a proper adapter registry and versioning system rather than manual file management. If adapters need to be swapped in and out of a production serving layer (like vLLM's multi-LoRA support) reliably, that deployment architecture is worth designing deliberately rather than debugging one incident at a time.
Frequently asked questions
Why does my LoRA adapter say target modules not found?
This means adapter_config.json lists module names (like q_proj or v_proj) that don't exist under those names in the base model you're loading. This happens when the base model architecture differs from the one used at training time, even if the model sizes and names look similar.
Can I load a LoRA adapter trained with an older PEFT version?
Usually yes, but not always. PEFT maintains reasonable backward compatibility, but major version jumps have changed state_dict key naming in the past. If loading fails with missing or unexpected key errors, try installing the exact PEFT version used during training first to confirm compatibility.
What files do I need to load a LoRA adapter?
You need both adapter_config.json (describing rank, alpha, and target modules) and the weight file, typically adapter_model.safetensors or adapter_model.bin. Both should be produced together by calling save_pretrained on a PeftModel; having only one of the two will cause loading to fail.
Can I merge a LoRA adapter into a different base model than it was trained on?
No. The low-rank weight deltas are mathematically tied to the exact parameter shapes and representations of the base model used during training. Merging onto a different model produces nonsensical outputs even if the merge operation itself does not error out.
Size it properly next time
Free calculators that prevent this class of failure before you provision hardware.
LoRA Fine-Tuning Cost Calculator
Turn model size, dataset tokens, epochs, and rank into a GPU-hour and dollar estimate for a LoRA fine-tuning run on rented or owned hardware.
Free ToolQLoRA vs Full Fine-Tuning Cost Calculator
See the GPU memory footprint, GPU-hour requirement, and dollar cost gap between QLoRA and full fine-tuning for the same model size and dataset.
Free ToolOpen-Weight Model Selector
A 10-question assessment that matches your hardware budget, workload complexity, and operational maturity to the right open-weight model size class.
Free ToolFine-Tuning GPU-Hours Estimator
Estimate GPU-hours for LoRA, QLoRA, and full fine-tuning on the same model size and dataset, so you can compare method tradeoffs before choosing.
Related problems
Training loss not decreasing during fine-tuning
Training loss that stays flat is most often caused by a LoRA adapter that targets the wrong modules (missing q_proj/k_proj/v_proj/o_proj), a learning rate that is too low for LoRA or too high and bouncing, or labels that were never masked so the model is trying to learn the prompt tokens as if they were random noise. Check target_modules first, then the label mask, then the learning rate.
Training dataset format errors during fine-tuning
Dataset format errors happen because the trainer expects a specific schema (either a messages list of role/content dicts, or a prompt/completion pair, or a single text field) and your JSONL doesn't match it, because samples are missing an EOS token so the model never learns to stop generating, or because a fixed max_length silently truncates long examples and cuts off labels partway through the intended response. Confirm your exact schema against what SFTTrainer or your data collator expects before training.
401/403 Unauthorized pulling a gated model from HuggingFace
A 401 or 403 on a gated HuggingFace repo means the request reached the Hub but was rejected for authorization, not because the model does not exist. The three real causes are: you have not accepted the model's license on the web page with the account tied to your token, the token exists but was created without read access to gated repos, or the token is valid but was never actually passed to the download call (no HF_TOKEN in the environment, no login run). Fix by accepting the license, generating a token with the right scope, and exporting it where the client library will find it.
Model type or architecture not recognized when loading a new model
An unrecognized model type or architecture error means the checkpoint's config.json declares a model_type or architectures field that your installed version of transformers or vLLM has never heard of, because that model's implementation code was added in a later library release than the one you have installed. The fix is almost always to upgrade the serving library (transformers, vLLM, or both) to a version released on or after that model's launch, not to modify the checkpoint.
GuideLoRA vs QLoRA: Choosing the Right Fine-Tuning Method
LoRA vs QLoRA for enterprise fine-tuning: rank and alpha choices, real VRAM math by model size, and when each method actually wins.
GuideFine-Tuning Failure Modes: What Actually Goes Wrong
Fine-tuning failure modes that actually derail enterprise projects: catastrophic forgetting, eval overfitting, data leakage, and how to catch each one.
GuideThe Model Upgrade Migration Playbook
A playbook for upgrading production LLMs: re-evaluation, prompt regression testing, rollback planning, and avoiding silent quality regressions.
GuideSecuring Model Weights in the Enterprise
Secure model weights end to end: custody controls, encryption at rest, access policies, and exfiltration prevention for regulated AI deployments.
Still stuck, or tired of fighting your own infrastructure?
Netray deploys and operates on-prem AI for regulated manufacturers and defense suppliers. We have debugged this stack in production, on air-gapped networks, at scale.