# Why Ollama silently forgets earlier turns, and how to fix context truncation > Source: https://netray.co/troubleshoot/ollama-context-length-truncation > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: On-Prem AI Troubleshooting ## Short answer Ollama silently truncates conversation history once the total tokens exceed num_ctx, which defaults to a relatively small value in many client configurations, dropping the oldest turns without any error or warning to the user or the calling application. The fix is to explicitly set num_ctx to a value that matches both your actual conversation length needs and the model's supported maximum, and to monitor token counts rather than assuming the full history is always sent. ## Error Title Ollama silently truncates earlier conversation turns ## Error Signature model responses ignore earlier context after a long conversation, as if it forgot the beginning ## Alternate Signatures - num_ctx default of 2048 silently drops earlier turns - ollama truncates context without warning - increasing num_ctx has no effect on remembered history ## Category local-runtime ## Stack - ollama - llama-cpp ## Affects Ollama and llama.cpp-based chat applications using default or unset num_ctx values, especially multi-turn conversational use cases ## Quick Fix ### Fix it in 60 seconds **Steps:** - Check the current num_ctx being used by inspecting the Modelfile or the API request parameters: it defaults to a modest value unless explicitly overridden. - Explicitly set num_ctx in your API call or Modelfile to a value that fits your target conversation length, for example 8192 or higher for long conversations. - Confirm the model itself actually supports that context length; setting num_ctx above the model's trained maximum does not add real capability, it can degrade output quality. - Recalculate expected VRAM usage after increasing num_ctx, since a larger context directly increases KV cache memory. - Add client-side logic to count tokens and warn or summarize older turns before they are silently dropped, rather than relying on the runtime to signal truncation. ## Symptoms - The model appears to forget instructions, names, or facts established earlier in a long conversation - Behavior changes noticeably once a conversation crosses a certain length, without any explicit error from the API - Increasing the visible conversation length in your application does not change the model's apparent memory - The same system prompt stops being followed after enough turns have accumulated ## Causes ### num_ctx left at its default value **Likelihood:** most-common **Explanation:** Ollama and many client wrappers apply a default context window unless the caller explicitly overrides it; once the running total of system prompt, conversation history, and new input exceeds that window, the runtime truncates from the oldest end of the context silently, with no error surfaced to the API caller or end user. **Fix:** Explicitly set num_ctx in the API request options or in a custom Modelfile PARAMETER num_ctx line to a value large enough for your expected conversation length, rather than relying on the default. **Commands:** - curl http://localhost:11434/api/generate -d '{"model": "llama3.2", "prompt": "hi", "options": {"num_ctx": 8192}}' ### Application layer not tracking or trimming conversation length itself **Likelihood:** common **Explanation:** Many chat applications simply append every new turn to a growing history array and send the whole thing on every request, trusting the model runtime to handle length; once that grows past num_ctx, the runtime's silent truncation becomes the application's de facto memory management strategy, which produces inconsistent behavior depending on exactly how much text was in earlier turns. **Fix:** Implement explicit token counting and either summarize or drop older turns deliberately in the application layer, so truncation is a controlled, visible decision rather than an accidental side effect of hitting num_ctx. **Commands:** ### num_ctx set higher than the model's actual trained context length **Likelihood:** occasional **Explanation:** Setting num_ctx above the maximum context length the model was actually trained or fine-tuned on does not grant real additional memory; instead it can produce degraded, less coherent output on the tokens beyond the model's real training window, which looks similar to truncation but is actually a capability limit rather than a configuration bug. **Fix:** Check the model's documented maximum context length in its model card or config and keep num_ctx at or below that value rather than assuming a larger number is always safe. **Commands:** ### KV cache memory limit forcing an effective cap below the requested num_ctx **Likelihood:** occasional **Explanation:** Even if num_ctx is set to a large value, if VRAM cannot hold the resulting KV cache alongside the model weights, some runtimes and configurations will fail to allocate it or fall back to a smaller effective window, which reintroduces truncation-like symptoms despite the configuration looking correct on paper. **Fix:** Verify available VRAM against the KV cache size implied by your num_ctx setting, and reduce num_ctx or move to a smaller quantization if the requested context does not actually fit in memory. **Commands:** - nvidia-smi --query-gpu=memory.used,memory.total --format=csv ### System prompt growing unboundedly and crowding out conversation history **Likelihood:** rare **Explanation:** In applications that append tool definitions, retrieved documents, or accumulating instructions to the system prompt on every turn, the system prompt itself can grow large enough to consume most of the context budget, leaving very little room for actual conversation history even with a generously sized num_ctx. **Fix:** Audit what is actually being sent as the system prompt on each request, and cap or periodically prune anything that grows unbounded (like an ever-expanding tool list or retrieved context) separately from the num_ctx setting. **Commands:** ## Diagnostics ### Check the num_ctx actually used in a request **Command:** curl http://localhost:11434/api/show -d '{"name": ""}' **Interpretation:** Shows the model's default parameters including context length; compare this against what your application explicitly passes in the options field of each request. ### Count tokens in the full request payload **Command:** python -c "import tiktoken; enc = tiktoken.get_encoding('cl100k_base'); print(len(enc.encode(open('payload.txt').read())))" **Interpretation:** If the token count of your full prompt plus history exceeds the configured num_ctx, truncation is definitely happening on that request, even if the exact tokenizer differs slightly from the target model's own. ### Check KV cache memory against VRAM **Command:** nvidia-smi --query-gpu=memory.used,memory.total --format=csv **Interpretation:** If VRAM is nearly full at your target num_ctx, the runtime may be capping context below what you requested to avoid an out-of-memory failure. ## Prevention - Always set num_ctx explicitly rather than relying on the runtime default, and document the chosen value alongside the reason for it. - Implement application-level token counting and deliberate history trimming or summarization instead of depending on silent runtime truncation. - Keep num_ctx at or below the model's actual trained context length rather than assuming larger is always better. - Monitor conversation length in production and alert when typical usage approaches your configured num_ctx limit. ## When To Escalate If your application genuinely needs long-running conversations or large retrieved-document context beyond what a single model's practical context window and your VRAM budget can support, that is an architecture problem calling for retrieval-based context management or a model with a larger native context window, not something you can configuration-tune your way out of indefinitely. ## Faq ### Does Ollama warn me when it truncates context? **Answer:** No. Truncation happens silently inside the runtime with no error or warning surfaced to the API response, which is exactly why it is such a common source of confusing the model forgot bug reports. ### What is Ollama's default num_ctx? **Answer:** It varies by version and how a model was pulled or configured, but it has historically defaulted to a modest value well below what many models actually support. Never assume a specific default; always set num_ctx explicitly for anything beyond casual local testing. ### Will setting a very large num_ctx always improve results? **Answer:** No. Beyond the model's real trained context length, a larger num_ctx does not add genuine memory and can degrade coherence. It also directly increases KV cache memory usage, so an unnecessarily large value can cause out-of-memory failures for no quality benefit. ## Search terms ollama num_ctx, ollama forgets context, ollama context truncation, ollama silent truncation, num_ctx default, ollama conversation memory --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json