# LLM Response Cache Token Savings Calculator

> Source: https://netray.co/tools/llm-response-cache-token-savings-calculator
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Free Tools

## Short answer

See how much a result cache and prompt prefix caching cut your monthly LLM API bill when many ERP questions repeat.

## Category

ai-automation

## Audience

- CIO
- IT manager
- FinOps

## Tool

**Kind:** calculator

**Currency:** $

**Inputs:**

#### Queries per day

**Key:** queries_day

**Type:** number

**Default:** 4000

**Min:** 0

**Step:** 100

**Unit:** queries

#### Days per month

**Key:** days

**Type:** number

**Default:** 30

**Min:** 1

**Max:** 31

**Step:** 1

**Unit:** days

#### Input tokens per query

**Key:** in_tokens

**Type:** number

**Default:** 3500

**Min:** 0

**Step:** 100

**Unit:** tokens

**Help:** System prompt, schema context, retrieved rows and the question.

#### Output tokens per query

**Key:** out_tokens

**Type:** number

**Default:** 500

**Min:** 0

**Step:** 50

**Unit:** tokens

#### Input price

**Key:** price_in

**Type:** number

**Default:** 3

**Min:** 0

**Step:** 0.1

**Unit:** $ per 1M tokens

#### Output price

**Key:** price_out

**Type:** number

**Default:** 15

**Min:** 0

**Step:** 0.5

**Unit:** $ per 1M tokens

#### Result cache hit rate

**Key:** hit

**Type:** slider

**Default:** 30

**Min:** 0

**Max:** 90

**Step:** 5

**Unit:** %

**Help:** Share of queries answered from a stored answer with no model call.

#### Static prefix share of input

**Key:** static_share

**Type:** slider

**Default:** 60

**Min:** 0

**Max:** 95

**Step:** 5

**Unit:** %

**Help:** Instructions and schema that are identical across calls.

#### Prefix cache discount on input

**Key:** prefix_discount

**Type:** number

**Default:** 90

**Min:** 0

**Max:** 100

**Step:** 5

**Unit:** %

**Help:** Check your provider's current cached-input price. Self-hosted engines get this via prefix caching in the serving layer.

**Outputs:**

#### Monthly cost without caching

**Key:** base

**Expr:** queries_day * days * (in_tokens * price_in + out_tokens * price_out) / 1000000

**Format:** currency

#### Cost after result cache only

**Key:** after_result

**Expr:** base * (1 - hit / 100)

**Format:** currency

#### Extra saving from prefix cache

**Key:** prefix_saving

**Expr:** queries_day * days * (1 - hit / 100) * in_tokens * static_share / 100 * prefix_discount / 100 * price_in / 1000000

**Format:** currency

#### Monthly cost with both caches

**Key:** optimized

**Expr:** after_result - prefix_saving

**Format:** currency

**Primary:** true

#### Total monthly saving

**Key:** saving

**Expr:** base - optimized

**Format:** currency

**Primary:** true

#### Saving as share of bill

**Key:** saving_pct

**Expr:** if(gt(base, 0), saving / base * 100, 0)

**Format:** percent

**Chart:**

#### Monthly LLM cost vs result cache hit rate

**Type:** line

**X:** hit

**From:** 0

**To:** 90

**Steps:** 18

**Series:**

- base
- optimized

**Disclaimer:** Prices change often and cache write fees are ignored. Cached answers can go stale: ERP data such as stock or order status needs short expiry or invalidation.

## Content

**Intro:** ERP chat tends to repeat itself: the same open-order, stock and aging questions arrive every morning. Two caches can cut spend: a result cache that skips the model, and prefix caching that discounts the repeated instruction and schema text.

**Sections:**

#### How to use it

**Body:** Pull a week of chat logs and count how many questions are repeats after normalization. That is your plausible hit rate. Enter token counts from your gateway logs, not estimates, since retrieved rows often dominate input. Set the static prefix share to the portion of the prompt that is byte-identical across calls. Then check your provider's cached-input price, or whether your self-hosted serving engine enables prefix caching.

#### The formula

**Body:** Base cost = queries x (input tokens x input price + output tokens x output price) / 1,000,000. Result caching multiplies by (1 - hit rate). Prefix caching then discounts the static share of input on the queries that still reach the model: saving = remaining queries x input tokens x static share x discount x input price. Both effects stack, but the second applies only to misses.

**Bullets:**

- Hit rate is the biggest lever.
- Prefix caching needs identical leading text.
- Put the changing question last in the prompt.

#### Cautions for ERP data

**Body:** A cached answer to 'what is on hand for part X' is wrong the moment stock moves. Key the cache by user role and question, set an expiry per data type, and never share cached answers across users with different row-level permissions. Dashboards of slow-changing data, policy questions and report definitions cache well; live inventory and order status do not. Measure answer correctness on cache hits, not only the hit rate.

**Cta:** Netray designs ERP chat with role-aware caching and visible queries. If you want to cut inference cost without leaking data across roles, talk to us.

**Faq:**

#### What is a realistic hit rate?

**Answer:** It varies widely with how repetitive your users are. Measure normalized repeats in a week of real logs before assuming a number; many deployments see a modest share, not a majority.

#### Is semantic caching safe for ERP?

**Answer:** Only with permission-aware keys and short expiry for live data. Similarity matching can return an answer to a nearly identical question with different filters, so test for that.

#### Does this apply to self-hosted models?

**Answer:** Result caching does. Prefix caching exists in engines such as vLLM, where it reduces compute rather than a bill, so use the discount as a rough proxy for GPU time saved.

#### Why is prefix saving only on misses?

**Answer:** Cache hits never call the model, so there is no prompt to discount. The prefix saving applies to the remaining calls.

## Search terms

llm cache savings calculator, prompt caching cost calculator, semantic cache hit rate savings, reduce llm api cost, erp chatbot token cost reduction, prefix caching savings, llm cost optimization calculator

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json