Why llama.cpp fails to load a GGUF file, and how to fix it
error loading model: unexpectedly reached end of file
Also appears as
- gguf_init_from_file: invalid magic number
- llama_model_load: error loading model: missing tensor
- failed to load model 'model.gguf'
Short answer
A GGUF load failure in llama.cpp is almost always one of three things: the file was truncated or corrupted during download, the file uses a quantization or metadata format newer than your llama.cpp build supports, or the model was split into multiple GGUF shards and only some of them were downloaded. Verify the file size and checksum first, then check your llama.cpp version against the GGUF version the file requires.
Affects: Any llama.cpp build or Ollama version loading a GGUF file, most common with large downloads and split or sharded models
Fastest path to a loading model
- 1Check the downloaded file size against the size listed on the source; a mismatch means a truncated download.
- 2Re-download the file with a tool that resumes and verifies, such as huggingface-cli download or wget -c, rather than a browser download for large files.
- 3If the model card lists multiple split GGUF files (part-00001-of-00003 style names), confirm every shard is present in the same directory.
- 4Update llama.cpp or Ollama to the latest release, since older builds cannot read newer GGUF metadata versions or newer quant types.
- 5Verify the file with a checksum if one is published, and only then retry loading.
How to confirm this is your problem
- Loading fails immediately with an end-of-file or truncated read error rather than a clear file-not-found error
- The error mentions an invalid magic number or unrecognized GGUF version
- The model loads on one machine's llama.cpp build but fails on another with an older version installed
- Downloading again from scratch fixes the problem, implying the first download was incomplete
Root causes and fixes
Corrupted or truncated download
GGUF files for larger models are many gigabytes, and interrupted downloads, flaky connections, or browser downloads that silently stop early leave a file that looks present on disk but is missing data at the end; llama.cpp's parser reads a fixed structure and hits an unexpected end of file when the tensor data runs short.
Fix: Compare the file size on disk against the expected size from the source, and re-download using a tool built for large, resumable transfers rather than a plain browser download.
ls -la model.gguf huggingface-cli download <repo> <file> --local-dir . wget -c <url>
GGUF format or quant type newer than the installed llama.cpp/Ollama version supports
The GGUF format and the set of supported quantization types have evolved across llama.cpp releases; a file quantized with a newer scheme or metadata version cannot be parsed by an older llama.cpp binary, which reports it as an invalid or unsupported file rather than explaining the version mismatch.
Fix: Update llama.cpp (rebuild from the latest source) or Ollama to the latest release, since GGUF forward compatibility depends entirely on the reading binary supporting the format the file was written with.
git pull && cmake --build build --config Release ollama --version
Split GGUF model missing one or more shard files
Very large models are distributed as multiple GGUF files named with a part index, and llama.cpp needs every shard present in the same directory to reconstruct the full tensor set; downloading only the first file, which is common when someone grabs just the file that looks like the main one, produces a load failure once the parser reaches data that lives in a missing shard.
Fix: Download every split file listed on the model page, keep them in the same directory with their original names, and load by pointing at the first shard, letting llama.cpp locate the rest automatically.
huggingface-cli download <repo> --local-dir . --include '*.gguf'
Wrong file passed as the model path
Pointing llama.cpp at a non-GGUF file (a raw safetensors checkpoint, a README, or a partially extracted archive) with a .gguf extension added by mistake produces the same generic load error, since the parser fails the same way whether the content is corrupt or simply not GGUF at all.
Fix: Confirm the file is actually a GGUF file by checking its header bytes, and re-download the correct converted GGUF artifact rather than a raw model checkpoint.
xxd model.gguf | head -1
Filesystem or disk issue causing silent data corruption
On rare occasions a failing disk, a network filesystem hiccup, or an interrupted copy between machines corrupts bytes in the middle of an otherwise complete-looking file, which can produce load errors that look identical to a truncated download but are not fixed by simply checking the file size.
Fix: Verify the file against a published checksum if available, and copy it fresh from the original source to a different disk to rule out local storage corruption.
sha256sum model.gguf
Diagnostic commands
Compare file size to expected size
ls -la model.gguf
A size noticeably smaller than what the source lists is a strong signal of a truncated download; re-download rather than debugging further.
Check the GGUF magic bytes
xxd model.gguf | head -1
A valid GGUF file starts with the ASCII bytes GGUF; anything else means the file is not a GGUF file at all, or is corrupted at the very start.
Check installed llama.cpp/Ollama version
ollama --version
Compare against the release notes or model card date for the GGUF file; a much older runtime version is a common reason newer quant types fail to load.
Verify checksum if published
sha256sum model.gguf
A mismatch against the published hash confirms corruption somewhere in the download or transfer path, independent of file size.
Stopping it from happening again
- Always download large GGUF files with a resumable, verifying tool (huggingface-cli, wget -c, aria2c) instead of a browser tab.
- Check published file sizes and checksums as a standard step before attempting to load any new model file.
- Keep llama.cpp and Ollama reasonably current, especially before trying a model released after your current installed version.
- For split models, download and verify every shard together as one atomic step, not incrementally over time.
When this becomes an architecture problem
If you are repeatedly hitting corrupted downloads or version mismatches across an air-gapped or bandwidth-constrained environment, it is worth building an internal, checksum-verified model artifact store that your team pulls from, rather than each engineer re-downloading multi-gigabyte files from the public internet individually.
Frequently asked questions
How do I know if a GGUF file is split into multiple parts?
The filenames follow a pattern like model-00001-of-00003.gguf, with the total shard count in the name. If the model card or repository lists more than one .gguf file for a single model, you need all of them present together, not just the first.
Can I fix a corrupted GGUF file without re-downloading it?
Generally no. GGUF corruption from a truncated or interrupted transfer means missing or wrong bytes, which cannot be reconstructed locally. Re-downloading, ideally with a tool that verifies integrity, is the reliable fix.
Does Ollama use the same GGUF format as raw llama.cpp?
Yes, Ollama is built on llama.cpp under the hood and uses the same GGUF file format, so a GGUF load failure has the same set of likely causes whether you hit it through Ollama or by running llama.cpp's binaries directly.
Size it properly next time
Free calculators that prevent this class of failure before you provision hardware.
Open-Weight Model Selector
A 10-question assessment that matches your hardware budget, workload complexity, and operational maturity to the right open-weight model size class.
Free ToolLLM Quantization Memory Savings Calculator
Compare FP16, FP8, and INT4 memory footprints for any model size and see how many fewer GPUs quantization requires to serve it.
Related problems
Ollama says a model was not found
Ollama model not found means the exact tag you requested, including the version suffix after the colon, does not exist locally or in the registry. Either the tag has a typo, the model was never pulled, or a custom Modelfile references a FROM path that does not resolve on this machine. Run ollama list to see what is actually installed, then pull or fix the Modelfile.
Corrupted model checkpoint fails to load or loads with garbage weights
A corrupted checkpoint means the bytes on disk do not match the original artifact the model author published, whether from an interrupted download, a bad copy between systems, disk-level bit rot, or a failed write during a save operation. There is no reliable way to repair a corrupted deep learning checkpoint; the fix is always to verify the file against a known-good hash or size and re-obtain a clean copy, then build a verification step into your pipeline so the same failure does not silently recur.
HuggingFace model download is extremely slow or stalls partway through
Slow or stalled HuggingFace downloads are usually caused by huggingface_hub's default transfer path not using parallel chunked downloads, a corporate proxy or firewall throttling or dropping long-lived connections, or genuinely insufficient bandwidth for a hundreds-of-gigabytes model. Enable hf_transfer for a much faster Rust-based parallel downloader, rely on the client's built-in resume behavior rather than restarting from zero, and for regulated or air-gapped sites, download once and mirror internally instead of pulling repeatedly over the internet.
GGUF quantization damages model output quality
GGUF quantization below roughly 4 bits per weight (Q2_K, Q3_K_S) trades accuracy aggressively for size and speed, and on smaller models or reasoning-heavy tasks this shows up as incoherent, repetitive, or factually unreliable output. Q4_K_M and Q5_K_M are the widely used sweet spots that keep most of the quality of the full-precision model while still cutting memory roughly in half or more, and Q2/Q3 should be reserved for cases where fitting in VRAM matters more than output quality.
GuideThe 2026 Open-Weight LLM Landscape: A Practical Map
A practical map of the 2026 open-weight LLM landscape: model families, license terms, and which model fits your VRAM budget and use case.
GuideLLM Quantization: AWQ vs GPTQ vs FP8 vs GGUF
AWQ, GPTQ, FP8, and GGUF compared for production LLM serving: memory savings, throughput impact, quality loss, and which format fits which deployment.
Still stuck, or tired of fighting your own infrastructure?
Netray deploys and operates on-prem AI for regulated manufacturers and defense suppliers. We have debugged this stack in production, on air-gapped networks, at scale.