# Model Context Window Planner: Budget Every Token in Your Prompt

> Source: https://netray.co/tools/model-context-window-planner
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Free Tools

## Short answer

Allocate a fixed context window across system prompt, retrieved chunks, conversation history, and reserved output, then see exactly how much retrieval headroom is left.

## Category

on-prem-ai

## Tool

**Kind:** calculator

**Inputs:**

#### Model context window

**Key:** context_window_tokens

**Type:** select

**Default:** 32768

**Unit:** tokens

**Help:** The total token budget shared by prompt and response. Self-hosted open-weight models often run 8K-128K.

**Options:**

##### 8K window (8,192 tokens)

**Value:** 8192

##### 16K window (16,384 tokens)

**Value:** 16384

##### 32K window (32,768 tokens)

**Value:** 32768

##### 128K window (128,000 tokens)

**Value:** 128000

##### 200K window (200,000 tokens)

**Value:** 200000

#### System prompt and tool definitions

**Key:** system_prompt_tokens

**Type:** number

**Default:** 1200

**Min:** 50

**Max:** 30000

**Step:** 50

**Unit:** tokens

**Help:** Instructions, persona, output schema, and JSON tool specs. Agent tool definitions grow faster than teams expect.

#### Average tokens per retrieved chunk

**Key:** avg_doc_tokens

**Type:** number

**Default:** 1200

**Min:** 100

**Max:** 20000

**Step:** 50

**Unit:** tokens

**Help:** Size of one retrieved passage after chunking. ERP work instructions and specs commonly run 800-2,000.

#### Chunks retrieved per query

**Key:** chunks_retrieved

**Type:** number

**Default:** 12

**Min:** 1

**Max:** 200

**Step:** 1

**Unit:** chunks

**Help:** Top-k passed to the model after reranking. More is not better once recall saturates.

#### Conversation turns kept in history

**Key:** conversation_turns

**Type:** number

**Default:** 6

**Min:** 0

**Max:** 100

**Step:** 1

**Unit:** turns

**Help:** How many prior exchanges you replay verbatim before summarizing or truncating.

#### Average tokens per stored turn

**Key:** avg_turn_tokens

**Type:** number

**Default:** 350

**Min:** 20

**Max:** 8000

**Step:** 10

**Unit:** tokens

**Help:** A user message plus its assistant reply. Verbose tool outputs inflate this quickly.

#### Reserved output tokens

**Key:** reserved_output_tokens

**Type:** number

**Default:** 1500

**Min:** 50

**Max:** 32000

**Step:** 50

**Unit:** tokens

**Help:** Space held back for the response. Reserve too little and answers get truncated mid-sentence.

**Outputs:**

#### Retrieval payload

**Key:** retrieval_tokens

**Expr:** avg_doc_tokens * chunks_retrieved

**Format:** number

**Description:** Tokens consumed by retrieved context on a typical query.

#### Conversation history

**Key:** history_tokens

**Expr:** conversation_turns * avg_turn_tokens

**Format:** number

**Description:** Tokens consumed by replayed prior turns.

#### Total context consumed

**Key:** total_prompt_tokens

**Expr:** system_prompt_tokens + retrieval_tokens + history_tokens + reserved_output_tokens

**Format:** number

**Description:** Everything that must fit inside the window, including the reserved response space.

**Primary:** true

#### Context window utilization

**Key:** window_utilization_pct

**Expr:** total_prompt_tokens / max(context_window_tokens, 1) * 100

**Format:** percent

**Description:** Share of the window consumed. Above 90% you are one long document away from an overflow error.

**Primary:** true

#### Additional chunks that still fit

**Key:** headroom_chunks

**Expr:** max(0, floor((context_window_tokens * 0.9 - total_prompt_tokens) / max(avg_doc_tokens, 1)))

**Format:** number

**Description:** Extra retrieved passages you can add while holding a 10% safety margin.

**Disclaimer:** Estimates only. Tokenizers differ across model families, so real counts can vary 10-20% for the same text. Always validate against the tokenizer your deployed model actually uses.

## Content

**Intro:** This free model context window planner allocates a fixed token budget across system instructions, retrieved documents, conversation history, and reserved output, and it is built for AI engineers and solution architects designing RAG and agent systems. Enter your model's window, prompt sizes, retrieval depth, and history policy, and the tool returns total context consumed, window utilization, and how many additional chunks still fit inside a safety margin. Context overflow is the single most common cause of silent quality failures in enterprise RAG: the system does not error, it quietly drops the passage that held the answer.

**Sections:**

#### How the budget is calculated

**Body:** Every component competes for the same window. The planner sums four claims: the system prompt and tool definitions, the retrieval payload, the replayed conversation history, and the output space you hold in reserve. With the defaults, a 1,200-token system prompt plus twelve 1,200-token chunks plus six 350-token turns plus 1,500 reserved output tokens totals 19,200 tokens, which is about 59% of a 32,768-token window. Headroom is then computed against 90% of the window rather than 100%, leaving a deliberate safety margin, so roughly eight more chunks would still fit. Push past that margin and a single unusually long document will trigger truncation or an outright error.

#### Planning benchmarks that hold up in production

**Body:** These reference points come from enterprise RAG deployments over ERP documentation, engineering specifications, quality records, and service histories. They are starting positions to tune against an evaluation set, not universal constants, and the right values differ meaningfully between a parts-lookup assistant and a specification-interpretation tool. The pattern we see repeatedly is teams over-retrieving early because more context feels safer, then discovering that precision fell and latency doubled without any accuracy gain. Establish a golden question set first, then move one variable at a time and measure. Guessing at these numbers is how a promising pilot quietly becomes an unreliable one.

**Bullets:**

- Retrieval quality typically saturates between 5 and 15 well-reranked chunks; adding more usually hurts precision and always raises cost.
- Keep total context utilization under 70% for interactive assistants so an outlier document does not break the request.
- Reserve at least 1,000-2,000 output tokens for explanatory answers, and far more for code or structured document generation.
- Summarize conversation history after roughly six turns instead of replaying it verbatim; it holds quality while cutting tokens sharply.

#### Interpreting utilization and headroom

**Body:** Utilization under 50% means you are leaving retrieval quality on the table and could pass more evidence or richer instructions. Between 50% and 75% is the comfortable operating band for most enterprise assistants. Above 85% you are in fragile territory, and you should either move to a larger window, shrink chunk size, or compress history. Also watch the effect of long context on accuracy: many models show measurable degradation on facts buried in the middle of a very large prompt, so a 128K window is not a licence to skip reranking. If headroom returns zero, your current configuration cannot serve a worst-case query at all and needs restructuring before launch.

#### How Netray tunes context for enterprise workloads

**Body:** Netray builds retrieval systems over manufacturing content that is messy by nature: revision-controlled work instructions, engineering drawings, supplier certifications, and decades of ERP transaction history in SyteLine or LN. We tune chunk boundaries to document structure rather than fixed character counts, add reranking so the top ten passages genuinely deserve their place, and implement history summarization that preserves decisions while discarding noise. For air-gapped and ITAR-constrained customers we do all of this with open-weight models running inside your network. Engagements usually start by measuring answer quality against a golden question set before any tuning begins.

**Cta:** Get a context and retrieval architecture tuned to your documents, your model, and your accuracy targets.

**Faq:**

#### Does a bigger context window remove the need for good retrieval?

**Answer:** No. Larger windows reduce hard truncation failures but do not fix relevance. Models frequently show weaker recall for facts positioned in the middle of very long prompts, and every extra token raises both latency and cost. In practice a well-reranked ten-chunk prompt beats a hundred-chunk dump on both accuracy and speed. Treat a large window as insurance against outliers, not as a substitute for chunking and reranking work.

#### How should I size chunks for ERP and engineering documents?

**Answer:** Chunk on structure rather than character counts wherever possible: one work-instruction step, one specification clause, one service history entry. That usually lands between 600 and 1,500 tokens. Overlap of 10-20% preserves context across boundaries. Fixed-size chunking splits tables and numbered procedures mid-thought, which is why so many first-generation RAG pilots over manufacturing content return confident but incomplete answers.

#### What happens if my prompt exceeds the context window?

**Answer:** Behavior varies and none of the options are good. Some APIs return a hard error, some silently truncate from the front, and some frameworks drop the oldest history without telling you. Front truncation is the most dangerous because it removes your system prompt, so the model loses its instructions and output schema while still producing fluent text. Always validate token counts before the call and fail loudly rather than truncating.

## Search terms

context window calculator, LLM context window planner, token budget calculator, RAG chunk sizing context window, prompt token budget planning, how many chunks fit in context window, context length planning LLM

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json