RAG Context Window Budget Calculator: Tokens and Cost Per Query
This free RAG context window budget calculator allocates your model's token window across system prompt, retrieved chunks, and conversation history, then converts the result into cost per query and projected daily spend, and it is built for AI engineers and platform leads tuning production retrieval systems. Enter your context window, prompt components, retrieval depth, and token pricing, and the tool returns total context consumed, window utilization, and a concrete cost figure per query. Retrieval depth is the single input most teams tune without ever seeing its direct effect on both accuracy and cost side by side, which is exactly what this calculator is built to show.
Your numbers
Total token budget shared by prompt and response for your chosen model.
Instructions, persona, citation format, and any tool specs the assistant carries on every call.
Top-k passages passed to the model after reranking, not the raw candidate count.
Size of one retrieved passage, set by your chunking strategy.
Prior exchanges replayed verbatim before summarization or truncation kicks in.
A user message plus its assistant reply, including any inline citations.
Space held back for the generated answer, including citations.
Applies to system prompt, retrieved chunks, and history combined.
Applies to the generated answer only.
Total RAG queries per day across all users, used to project daily spend.
Your results
Estimates only. Real tokenizers vary by model family and pricing changes frequently. Validate against your deployed model's tokenizer and current rate card before finalizing a budget.
Get your full RAG context and cost model
We will email you a personalized token budget and cost-per-query breakdown across retrieval depth scenarios, and a Netray AI architect will follow up on your retrieval tuning.
No spam. Your results stay private. Unsubscribe anytime.
How the budget and cost are calculated
The calculator sums four claims on the context window: the system prompt, the retrieval payload (chunks retrieved times chunk size), the replayed conversation history, and reserved output space. With the defaults, 800 system prompt tokens plus 3,200 retrieval tokens (8 chunks at 400 tokens) plus 1,200 history tokens (4 turns at 300 tokens) plus 800 reserved output tokens totals 6,000 tokens, about 18% of a 32,768-token window. Cost per query prices the input components (everything except reserved output) at the input rate and the output allocation at the output rate: at $3 and $15 per million tokens, that comes to roughly $0.0276 per query, or about $138 per day at 5,000 queries.
Why retrieval depth is the highest-leverage lever here
Chunks retrieved multiplies directly into both context consumption and cost, and it is usually the input teams tune the least deliberately, defaulting to a round number like 10 or 20 and never revisiting it against measured recall. Doubling chunks retrieved from 8 to 16 roughly doubles retrieval tokens and pushes cost per query up proportionally, often without a matching accuracy gain once retrieval recall has already saturated. The right number is found empirically against a golden question set, not chosen as a safe-feeling default.
- Retrieval quality typically saturates between 5 and 15 well-reranked chunks for most enterprise queries.
- Doubling chunks retrieved roughly doubles retrieval token cost with diminishing accuracy return past saturation.
- Reranking lets you retrieve fewer, higher-quality chunks, cutting cost while often improving accuracy.
- History replay cost compounds across a long conversation; summarizing after a few turns caps this growth.
Reading utilization and cost together
Utilization under 50% usually means there is room to retrieve more context if recall testing shows it would help, though that also raises cost proportionally. Utilization above 85% is fragile territory where a single long document risks pushing a request into truncation or an outright overflow error. Cost per query is the number to defend to finance, and it should be evaluated against the value each query creates, not in isolation: a $0.03 query that resolves a five-minute ERP lookup is trivial, while the same cost multiplied across a poorly-scoped agent loop that fires redundant retrievals per user question adds up fast.
How Netray tunes context and cost together for enterprise RAG
Netray tunes retrieval depth against measured recall rather than intuition, using golden question sets built from real manufacturing content: ERP records, engineering specifications, and service history in SyteLine and LN. We add reranking so fewer, better chunks reach the model, implement history summarization that holds conversational quality while capping token growth, and validate the resulting cost model against your actual query volume before it goes into a budget. For air-gapped customers we do this entirely with open-weight models running inside your network, converting a variable per-token bill into predictable infrastructure cost.
Frequently Asked Questions
How many chunks should I retrieve per query?
Start around 8-12 well-reranked chunks and validate against a golden question set rather than defaulting to a round number. Retrieval quality for most enterprise queries saturates in this range; adding more chunks beyond it usually raises cost and can hurt precision without improving recall. If you find you need 20 or more chunks to get good answers, the more likely fix is improving chunking and reranking quality rather than retrieving deeper.
Should I worry about cost per query or total daily cost?
Both, for different reasons. Cost per query tells you whether the unit economics make sense: compare it against the value a single answer creates. Total daily cost tells you whether the aggregate is sustainable and how it will scale with adoption. A query that costs three cents is trivial individually but becomes a meaningful budget line at hundreds of thousands of queries per day, which is why the daily projection matters just as much once a pilot moves toward broader rollout.
Does conversation history really need to be replayed in full every time?
Not indefinitely. Replaying every prior turn verbatim is simple to implement but its token cost grows linearly with conversation length, eventually crowding out retrieval budget or triggering context overflow. Most production systems summarize history after roughly four to six turns, preserving key decisions and facts while discarding verbose exchanges. This holds conversational quality reasonably well while capping token growth, which keeps both cost and window utilization predictable regardless of how long a session runs.
How does this calculator relate to the RAG chunking strategy calculator?
They cover adjacent decisions. The chunking strategy calculator sizes how many chunks your corpus produces and what they cost to embed once, upstream in the ingestion pipeline. This calculator sizes what happens at query time: how many of those chunks get retrieved and passed to the model, and what each query costs as a result. Use the chunking tool to design ingestion, then this tool to tune retrieval depth and validate the ongoing per-query cost that decision creates.
Get a context and cost model tuned to your retrieval depth, your model, and your actual query volume.
Related Tools
Model Context Window Planner
Allocate a fixed context window across system prompt, retrieved chunks, conversation history, and reserved output, then see exactly how much retrieval headroom is left.
On-Prem AILLM Token Cost Calculator
Turn request volume, prompt length, and per-million token pricing into a defensible monthly and annual LLM budget, including the effect of prompt caching.
On-Prem AIRAG Chunking Strategy Calculator
Turn corpus size, chunk length, and overlap into a concrete chunk count, embedding cost, and vector storage footprint before you build the ingestion pipeline.
Go Deeper
Enterprise RAG Architecture: The Full 2026 Blueprint
A practitioner's blueprint for enterprise RAG in 2026: ingestion, chunking, embedding, retrieval, rerank, generation, and the eval loop that keeps it honest.
RAG Cost Optimization: Cutting Spend Without Losing Accuracy
Practical RAG cost optimization: cheaper embedding models, caching, smaller rerankers, prompt compression, and when self-hosting beats API spend.
On-Prem LLM Deployment Architecture: Reference Guide
Reference architecture for on-prem LLM deployment: inference servers, GPU sizing, RAG pipelines, and security zones for regulated manufacturers.