# Why Ollama runs out of memory, and how to fix it > Source: https://netray.co/troubleshoot/ollama-out-of-memory > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: On-Prem AI Troubleshooting ## Short answer Ollama out of memory happens when the model's weights plus its KV cache exceed either GPU VRAM or system RAM, and the OOM killer or CUDA allocator terminates the process. The fix depends on which resource is exhausted: reduce quantization or context length for VRAM limits, or reduce concurrent model loads and context for system RAM limits. ## Error Title Ollama runs out of memory loading or running a model ## Error Signature Error: model requires more system memory (5.4 GiB) than is available (3.9 GiB) ## Alternate Signatures - CUDA error: out of memory - Error: llama runner process has terminated: signal: killed - ollama: Killed ## Category local-runtime ## Stack - ollama - cuda - nvidia-driver ## Affects Ollama on GPU and CPU-only hosts, most common when running models close to the size of available VRAM or RAM ## Quick Fix ### Fix it in 60 seconds **Steps:** - Check dmesg or the system logs for an OOM-killer entry to confirm whether this is system RAM or GPU VRAM exhaustion. - Run nvidia-smi to see current VRAM usage versus total before loading the model. - If VRAM is the constraint, switch to a smaller quantization (Q4_K_M instead of Q8_0 or FP16) or a smaller parameter count model. - Reduce num_ctx to lower the KV cache size, since context length directly multiplies memory usage per request. - If multiple models are loaded simultaneously, unload unused ones with ollama stop before loading a new one. ## Symptoms - Ollama reports a specific error stating the model needs more memory than is available before it even starts generating - The ollama process is killed abruptly with no application-level error, visible only as Killed in the terminal or an OOM entry in dmesg - A model that worked at a shorter context length fails once conversation history grows - Loading a second model while the first is still resident causes an immediate failure ## Causes ### Model plus KV cache exceeds VRAM **Likelihood:** most-common **Explanation:** GPU memory has to hold model weights, the KV cache for the current context, and some working memory for the forward pass all at once; KV cache size scales with batch size times context length times the number of layers and attention heads, so a model that fits at a short context can OOM once num_ctx or a long conversation grows the cache beyond free VRAM. **Fix:** Reduce num_ctx to the minimum your use case needs, or move to a smaller quantization so more VRAM headroom remains for the cache. **Commands:** - nvidia-smi --query-gpu=memory.used,memory.total --format=csv - ollama run --verbose ### System RAM insufficient for CPU-offloaded layers **Likelihood:** common **Explanation:** When a model does not fully fit in VRAM, Ollama offloads the remainder to system RAM; if total system RAM is also small relative to the model size, the offloaded portion itself cannot be allocated and the OS OOM killer terminates the process. **Fix:** Either free up system RAM by closing other processes, add swap as a temporary buffer, or choose a smaller model/quantization that fits the combined VRAM and RAM budget more comfortably. **Commands:** - free -h - dmesg | grep -i 'out of memory' ### Multiple models or concurrent requests loaded at once **Likelihood:** common **Explanation:** Ollama can keep more than one model resident in memory, and may also handle concurrent request contexts; each additional resident model or concurrent context adds its own KV cache on top of shared weight memory, which can exhaust VRAM even when a single model alone would fit comfortably. **Fix:** Unload models you are not actively using with ollama stop, and if serving concurrent users, explicitly manage or limit the number of parallel model instances Ollama keeps loaded. **Commands:** - ollama ps - ollama stop ### Very large context request in a single call **Likelihood:** occasional **Explanation:** A single API call that passes an unusually large prompt, well beyond typical usage, forces a correspondingly large KV cache allocation in one shot; if that allocation alone exceeds free VRAM, the request fails immediately regardless of how well-sized the base model is. **Fix:** Cap the maximum prompt size accepted by your application layer before it reaches Ollama, and validate token counts client-side rather than relying on the runtime to reject oversized requests gracefully. **Commands:** ### Memory fragmentation after many model swaps **Likelihood:** rare **Explanation:** Repeatedly loading and unloading different models in the same long-running process can, in rare cases, leave the CUDA allocator with fragmented free memory such that no single contiguous block large enough for the next model is available, even though the total free VRAM reported by nvidia-smi looks sufficient. **Fix:** Restart the Ollama service periodically in long-running deployments that swap models frequently, which resets the allocator to a clean state. **Commands:** - systemctl restart ollama ## Diagnostics ### Check for an OS-level OOM kill **Command:** dmesg | grep -i 'out of memory' **Interpretation:** A hit here means system RAM, not VRAM, was exhausted and the kernel killed the process; the fix is RAM or offload management, not GPU quantization alone. ### Check VRAM headroom before loading **Command:** nvidia-smi --query-gpu=memory.used,memory.total --format=csv **Interpretation:** If used memory is already high before you load your model, another process or a previously loaded model is consuming the budget you thought was free. ### Check system RAM availability **Command:** free -h **Interpretation:** Low available memory here, especially combined with high swap usage, confirms system RAM is the bottleneck for CPU-offloaded layers. ### See what Ollama currently has resident **Command:** ollama ps **Interpretation:** Multiple models listed as loaded simultaneously explains VRAM pressure that a single model's size alone would not predict. ## Prevention - Size models against free VRAM and RAM, not total capacity, leaving headroom for the KV cache at your real target context length. - Set an explicit, conservative num_ctx default in your application rather than trusting a large runtime default. - Monitor VRAM and system RAM continuously in production and alert before utilization approaches the ceiling, not after an OOM kill. - Limit how many distinct models can be resident simultaneously in memory-constrained deployments. ## When To Escalate If you are hitting memory limits with a right-sized quantization and a reasonable context length, the hardware is simply undersized for the models and concurrency you need, and the right next step is a proper capacity plan (bigger GPU, multiple GPUs, or a dedicated serving stack with admission control) rather than continuing to trim context length. ## Faq ### Does Ollama tell me how much memory a model needs before loading it? **Answer:** Ollama estimates memory requirements internally and will sometimes report the shortfall in its error message, but it does not always give a precise breakdown ahead of time. Checking nvidia-smi and free before and after loading is the most reliable way to see the actual footprint. ### Will adding swap fix Ollama out of memory errors? **Answer:** Swap can prevent an immediate OOM kill by giving the OS more room, but disk-backed swap is drastically slower than RAM, so a model that needs swap to run will generate tokens extremely slowly. Treat swap as an emergency buffer, not a real fix. ### Why does the same model OOM only sometimes? **Answer:** This is almost always a context length effect: shorter conversations fit in the available memory budget while longer ones grow the KV cache past it, so the same model and quantization can succeed or fail depending on how much conversation history is being sent. ## Search terms ollama out of memory, ollama requires more system memory, ollama killed, ollama vram oom, ollama context length memory, ollama cuda out of memory --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json