Prompt Cache Savings Calculator for Agent Tool Prefixes
Agents re-send the same system prompt and tool definitions on every call. If your provider offers prompt caching, that stable prefix can be billed far below the base price on a hit. This tool shows what that is worth.
Your numbers
System prompt plus tool definitions that are identical across calls.
User text, history and tool results.
Check your provider. Some discount cached reads heavily.
Some providers charge a premium on a cache miss that writes the prefix.
Your results
Pricing structures differ by provider and may include minimum cacheable sizes and time-to-live limits. Enter your provider's actual multipliers.
Daily savings by cache hit rate
Sweeps Cache hit rate from 0 to 100, holding your other inputs fixed.
LLM cost optimization chat
Share your usage pattern and we will point out the biggest cost levers. Optional.
Optional. The tool and every export work without this. No spam.
How to use it
Set the token size of the stable prefix: system prompt plus tool definitions that do not change between calls. Enter dynamic tokens per call and daily call volume. Then enter your provider's cache read and write multipliers and an expected hit rate. You get daily and monthly savings, the share of input spend saved and the break-even hit rate. The chart sweeps hit rate from zero to one hundred so you can see how sensitive savings are to cache misses.
Formula
Without caching, input cost is calls times total tokens times price. With caching, the prefix is billed at the read fraction on hits and at the write multiple on misses, while dynamic tokens stay at the base price. Break-even hit rate is (write multiple minus 1) divided by (write multiple minus read fraction): below it, the write premium on misses outweighs the discount on hits.
- Keep the prefix byte-identical and place dynamic content after it.
- Sort tool lists deterministically.
- Cache lifetimes are short, so low traffic lowers hit rate.
Common mistakes
Putting a timestamp or user name at the top of the prompt breaks the prefix and drops the hit rate to zero. Changing tool order between deployments or between users has the same effect. Also check that your prefix exceeds the provider's minimum cacheable length, and remember that caching reduces price and often latency but not the window space the prefix uses.
Frequently Asked Questions
What is a typical cache hit rate?
It depends on traffic. Steady, frequent traffic with an unchanging prefix can see high hit rates, while sporadic use lets cache entries expire between calls. Measure from your provider's usage fields and enter the real figure.
Does caching change answers?
No. It reuses computed state for an identical prefix, so the model sees the same input. It affects billing and latency, not the content of the response.
What breaks the cache?
Any change before the cached boundary: edited system prompt, reordered or modified tool definitions, injected dates or user-specific text near the top. Keep volatile content after the stable prefix.
Is this relevant for self-hosted models?
The concept is similar. Engines such as vLLM support automatic prefix caching, which saves GPU compute instead of dollars. The same prefix discipline applies, but the savings show up as throughput.
If you run private models instead, the equivalent is prefix caching in the serving engine. Netray can help tune that for your ERP agents.
Related Tools
MCP Tool Definition Context Token Overhead Calculator
Sizes the context window and input cost eaten by MCP tool schemas before the user says a word, and shows the what-if as you add tools.
AI Agents & AutomationAgent Loop Cost Per Task Calculator
Models input tokens that accumulate across tool calls in an agent loop and prices a single task and a month of tasks.
AI Agents & AutomationAgent Multi-Step Reliability Calculator
Shows how per-step success compounds across an agent workflow, what retries buy you and how many tool calls to expect.
On-Prem AILLM Token Cost Calculator
Turn request volume, prompt length, and per-million token pricing into a defensible monthly and annual LLM budget, including the effect of prompt caching.
More AI Agents & Automation tools
See all- Business Process Automation ROI Calculator
- AI Agent Use Case Prioritizer
- Document Processing Automation Savings Calculator
- AP Invoice Automation ROI Calculator
- Order Entry Automation Savings Calculator
- Quote-to-Cash Cycle Time Calculator
- Manufacturing AI Readiness Assessment
- RPA vs AI Agent Selector
- ERP AI Maturity Assessment