# RAG Context Window Budget Calculator: Tokens and Cost Per Query

> Source: https://netray.co/tools/rag-context-window-budget-calculator
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Free Tools

## Short answer

Allocate your context window across system prompt, retrieved chunks, and conversation history, then see window utilization and the real cost of every RAG query.

## Category

on-prem-ai

## Tool

**Kind:** calculator

**Inputs:**

#### Model context window

**Key:** context_window_tokens

**Type:** select

**Default:** 32768

**Unit:** tokens

**Help:** Total token budget shared by prompt and response for your chosen model.

**Options:**

##### 8K window (8,192 tokens)

**Value:** 8192

##### 16K window (16,384 tokens)

**Value:** 16384

##### 32K window (32,768 tokens)

**Value:** 32768

##### 128K window (128,000 tokens)

**Value:** 128000

##### 200K window (200,000 tokens)

**Value:** 200000

#### System prompt and tool definitions

**Key:** system_prompt_tokens

**Type:** number

**Default:** 800

**Min:** 50

**Max:** 30000

**Step:** 50

**Unit:** tokens

**Help:** Instructions, persona, citation format, and any tool specs the assistant carries on every call.

#### Chunks retrieved per query

**Key:** chunks_retrieved

**Type:** number

**Default:** 8

**Min:** 1

**Max:** 100

**Step:** 1

**Unit:** chunks

**Help:** Top-k passages passed to the model after reranking, not the raw candidate count.

#### Average tokens per chunk

**Key:** chunk_size_tokens

**Type:** number

**Default:** 400

**Min:** 50

**Max:** 4000

**Step:** 25

**Unit:** tokens

**Help:** Size of one retrieved passage, set by your chunking strategy.

#### Conversation turns kept in history

**Key:** history_turns

**Type:** number

**Default:** 4

**Min:** 0

**Max:** 50

**Step:** 1

**Unit:** turns

**Help:** Prior exchanges replayed verbatim before summarization or truncation kicks in.

#### Average tokens per stored turn

**Key:** avg_turn_tokens

**Type:** number

**Default:** 300

**Min:** 20

**Max:** 5000

**Step:** 10

**Unit:** tokens

**Help:** A user message plus its assistant reply, including any inline citations.

#### Reserved output tokens

**Key:** reserved_output_tokens

**Type:** number

**Default:** 800

**Min:** 50

**Max:** 16000

**Step:** 50

**Unit:** tokens

**Help:** Space held back for the generated answer, including citations.

#### Input price per 1M tokens

**Key:** input_price_per_1m

**Type:** number

**Default:** 3

**Min:** 0.01

**Max:** 200

**Step:** 0.05

**Unit:** $

**Help:** Applies to system prompt, retrieved chunks, and history combined.

#### Output price per 1M tokens

**Key:** output_price_per_1m

**Type:** number

**Default:** 15

**Min:** 0.01

**Max:** 500

**Step:** 0.25

**Unit:** $

**Help:** Applies to the generated answer only.

#### Queries per day

**Key:** queries_per_day

**Type:** number

**Default:** 5000

**Min:** 10

**Max:** 10000000

**Step:** 100

**Unit:** queries

**Help:** Total RAG queries per day across all users, used to project daily spend.

**Outputs:**

#### Retrieval payload

**Key:** retrieval_tokens

**Expr:** chunks_retrieved * chunk_size_tokens

**Format:** number

**Description:** Tokens consumed by retrieved context on a typical query.

#### Total context consumed

**Key:** total_prompt_tokens

**Expr:** system_prompt_tokens + retrieval_tokens + (history_turns * avg_turn_tokens) + reserved_output_tokens

**Format:** number

**Description:** Everything that must fit inside the window, including the reserved response space.

#### Context window utilization

**Key:** window_utilization_pct

**Expr:** total_prompt_tokens / max(context_window_tokens, 1) * 100

**Format:** percent

**Description:** Share of the window consumed. Above 85% you are close to overflow on longer documents.

#### Cost per query

**Key:** cost_per_query

**Expr:** (system_prompt_tokens + retrieval_tokens + (history_turns * avg_turn_tokens)) / 1000000 * input_price_per_1m + reserved_output_tokens / 1000000 * output_price_per_1m

**Format:** currency

**Description:** Blended input and output cost for one complete RAG query.

**Primary:** true

#### Projected daily cost

**Key:** daily_cost

**Expr:** cost_per_query * queries_per_day

**Format:** currency

**Description:** Total spend at your current query volume, before any growth.

**Disclaimer:** Estimates only. Real tokenizers vary by model family and pricing changes frequently. Validate against your deployed model's tokenizer and current rate card before finalizing a budget.

## Content

**Intro:** This free RAG context window budget calculator allocates your model's token window across system prompt, retrieved chunks, and conversation history, then converts the result into cost per query and projected daily spend, and it is built for AI engineers and platform leads tuning production retrieval systems. Enter your context window, prompt components, retrieval depth, and token pricing, and the tool returns total context consumed, window utilization, and a concrete cost figure per query. Retrieval depth is the single input most teams tune without ever seeing its direct effect on both accuracy and cost side by side, which is exactly what this calculator is built to show.

**Sections:**

#### How the budget and cost are calculated

**Body:** The calculator sums four claims on the context window: the system prompt, the retrieval payload (chunks retrieved times chunk size), the replayed conversation history, and reserved output space. With the defaults, 800 system prompt tokens plus 3,200 retrieval tokens (8 chunks at 400 tokens) plus 1,200 history tokens (4 turns at 300 tokens) plus 800 reserved output tokens totals 6,000 tokens, about 18% of a 32,768-token window. Cost per query prices the input components (everything except reserved output) at the input rate and the output allocation at the output rate: at $3 and $15 per million tokens, that comes to roughly $0.0276 per query, or about $138 per day at 5,000 queries.

#### Why retrieval depth is the highest-leverage lever here

**Body:** Chunks retrieved multiplies directly into both context consumption and cost, and it is usually the input teams tune the least deliberately, defaulting to a round number like 10 or 20 and never revisiting it against measured recall. Doubling chunks retrieved from 8 to 16 roughly doubles retrieval tokens and pushes cost per query up proportionally, often without a matching accuracy gain once retrieval recall has already saturated. The right number is found empirically against a golden question set, not chosen as a safe-feeling default.

**Bullets:**

- Retrieval quality typically saturates between 5 and 15 well-reranked chunks for most enterprise queries.
- Doubling chunks retrieved roughly doubles retrieval token cost with diminishing accuracy return past saturation.
- Reranking lets you retrieve fewer, higher-quality chunks, cutting cost while often improving accuracy.
- History replay cost compounds across a long conversation; summarizing after a few turns caps this growth.

#### Reading utilization and cost together

**Body:** Utilization under 50% usually means there is room to retrieve more context if recall testing shows it would help, though that also raises cost proportionally. Utilization above 85% is fragile territory where a single long document risks pushing a request into truncation or an outright overflow error. Cost per query is the number to defend to finance, and it should be evaluated against the value each query creates, not in isolation: a $0.03 query that resolves a five-minute ERP lookup is trivial, while the same cost multiplied across a poorly-scoped agent loop that fires redundant retrievals per user question adds up fast.

#### How Netray tunes context and cost together for enterprise RAG

**Body:** Netray tunes retrieval depth against measured recall rather than intuition, using golden question sets built from real manufacturing content: ERP records, engineering specifications, and service history in SyteLine and LN. We add reranking so fewer, better chunks reach the model, implement history summarization that holds conversational quality while capping token growth, and validate the resulting cost model against your actual query volume before it goes into a budget. For air-gapped customers we do this entirely with open-weight models running inside your network, converting a variable per-token bill into predictable infrastructure cost.

**Cta:** Get a context and cost model tuned to your retrieval depth, your model, and your actual query volume.

**Faq:**

#### How many chunks should I retrieve per query?

**Answer:** Start around 8-12 well-reranked chunks and validate against a golden question set rather than defaulting to a round number. Retrieval quality for most enterprise queries saturates in this range; adding more chunks beyond it usually raises cost and can hurt precision without improving recall. If you find you need 20 or more chunks to get good answers, the more likely fix is improving chunking and reranking quality rather than retrieving deeper.

#### Should I worry about cost per query or total daily cost?

**Answer:** Both, for different reasons. Cost per query tells you whether the unit economics make sense: compare it against the value a single answer creates. Total daily cost tells you whether the aggregate is sustainable and how it will scale with adoption. A query that costs three cents is trivial individually but becomes a meaningful budget line at hundreds of thousands of queries per day, which is why the daily projection matters just as much once a pilot moves toward broader rollout.

#### Does conversation history really need to be replayed in full every time?

**Answer:** Not indefinitely. Replaying every prior turn verbatim is simple to implement but its token cost grows linearly with conversation length, eventually crowding out retrieval budget or triggering context overflow. Most production systems summarize history after roughly four to six turns, preserving key decisions and facts while discarding verbose exchanges. This holds conversational quality reasonably well while capping token growth, which keeps both cost and window utilization predictable regardless of how long a session runs.

#### How does this calculator relate to the RAG chunking strategy calculator?

**Answer:** They cover adjacent decisions. The chunking strategy calculator sizes how many chunks your corpus produces and what they cost to embed once, upstream in the ingestion pipeline. This calculator sizes what happens at query time: how many of those chunks get retrieved and passed to the model, and what each query costs as a result. Use the chunking tool to design ingestion, then this tool to tune retrieval depth and validate the ongoing per-query cost that decision creates.

## Search terms

RAG context window calculator, context window budget, cost per query calculator, RAG token budget planning, retrieved chunks context window, LLM prompt cost RAG, context window utilization

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json