# LLM Model Routing Cost Savings Calculator

> Source: https://netray.co/tools/llm-model-routing-cost-savings-calculator
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Free Tools

## Short answer

Work out blended cost per request when a router sends most traffic to a small model and escalates failures to a large one.

## Category

on-prem-ai

## Audience

- CIO
- IT manager
- AI platform team

## Tool

**Kind:** calculator

**Currency:** $

**Inputs:**

#### Requests per month

**Key:** requests

**Type:** number

**Default:** 300000

**Min:** 0

**Step:** 10000

**Unit:** requests

#### Input tokens per request

**Key:** in_tokens

**Type:** number

**Default:** 1800

**Min:** 0

**Step:** 100

**Unit:** tokens

#### Output tokens per request

**Key:** out_tokens

**Type:** number

**Default:** 350

**Min:** 0

**Step:** 50

**Unit:** tokens

#### Small model input price

**Key:** small_in

**Type:** number

**Default:** 0.15

**Min:** 0

**Step:** 0.05

**Unit:** $ per 1M tokens

#### Small model output price

**Key:** small_out

**Type:** number

**Default:** 0.6

**Min:** 0

**Step:** 0.05

**Unit:** $ per 1M tokens

#### Large model input price

**Key:** large_in

**Type:** number

**Default:** 3

**Min:** 0

**Step:** 0.1

**Unit:** $ per 1M tokens

#### Large model output price

**Key:** large_out

**Type:** number

**Default:** 15

**Min:** 0

**Step:** 0.5

**Unit:** $ per 1M tokens

#### Share routed to small model

**Key:** small_share

**Type:** slider

**Default:** 70

**Min:** 0

**Max:** 100

**Step:** 5

**Unit:** %

#### Small model answers escalated

**Key:** escalate

**Type:** number

**Default:** 10

**Min:** 0

**Max:** 100

**Step:** 1

**Unit:** %

**Help:** Share of small-model answers that fail a check and are retried on the large model.

#### Router cost

**Key:** router_cost

**Type:** number

**Default:** 0.05

**Min:** 0

**Step:** 0.01

**Unit:** $ per 1,000 requests

**Help:** Classifier or embedding call used to decide the route.

**Outputs:**

#### Large model cost per request

**Key:** large_req

**Expr:** (in_tokens * large_in + out_tokens * large_out) / 1000000

**Format:** decimal

#### Small model cost per request

**Key:** small_req

**Expr:** (in_tokens * small_in + out_tokens * small_out) / 1000000

**Format:** decimal

#### Monthly cost, everything on large

**Key:** all_large

**Expr:** requests * large_req

**Format:** currency

#### Monthly cost with routing

**Key:** routed

**Expr:** requests * (small_share / 100 * (small_req + escalate / 100 * large_req) + (1 - small_share / 100) * large_req) + requests / 1000 * router_cost

**Format:** currency

**Primary:** true

#### Monthly saving

**Key:** saving

**Expr:** all_large - routed

**Format:** currency

**Primary:** true

#### Saving as share of large-only cost

**Key:** saving_pct

**Expr:** if(gt(all_large, 0), saving / all_large * 100, 0)

**Format:** percent

**Chart:**

#### Monthly cost vs share routed to the small model

**Type:** line

**X:** small_share

**From:** 0

**To:** 100

**Steps:** 20

**Series:**

- routed
- all_large

**Disclaimer:** Cost only. Routing quality is not modeled: measure answer quality per route on your own questions before shifting traffic.

## Content

**Intro:** Most ERP questions are simple lookups, and a small model answers them well. Routing the easy traffic to a cheap model and escalating failures to a larger one can cut cost, but only if escalations and router overhead are counted.

**Sections:**

#### How to use it

**Body:** Enter your monthly volume and average token counts from gateway logs. Enter each model's price per million tokens, whether from an API provider or your own cost per million tokens on owned GPUs. Set the share your router sends to the small model, and the share of those answers that fail a verification check and are retried on the large model. Include the cost of the routing step itself.

#### The formula

**Body:** Per-request cost = (input tokens x input price + output tokens x output price) / 1,000,000. Routed cost = requests x [ small share x (small cost + escalation x large cost) + (1 - small share) x large cost ] + router cost. The escalation term is why a high small share is not automatically cheap: every failed attempt is paid for twice.

**Bullets:**

- Escalation rate depends on the check you use.
- Retries also add latency.
- Router must be far cheaper than the savings.

#### Reading the result

**Body:** If saving is under about ten percent, routing adds complexity for little gain and a single mid-size model may be simpler to run. Savings are larger when the price gap between models is wide and your traffic is dominated by lookups. Validate with a labeled set of real questions per route. A small model that fails silently on aggregations or multi-table joins can cost far more in wrong answers than it saves in tokens.

**Cta:** Netray runs open-weight models of several sizes on customer GPUs and can help design routing with verification. Ask us if you want it benchmarked on your ERP questions.

**Faq:**

#### How do I set the escalation rate?

**Answer:** Run a labeled sample of real questions through the small model with your verification check, then count how many fail. Use that fraction, and re-measure after prompt or model changes.

#### Does this work for self-hosted models?

**Answer:** Yes. Replace the prices with your cost per million tokens, which is GPU cost per hour divided by tokens served per hour at realistic utilization.

#### What about latency?

**Answer:** Escalated requests take two model calls. Small-model answers are usually faster, so average latency often improves but the tail gets longer.

#### When is routing not worth it?

**Answer:** When the saving is small, traffic is low, or accuracy risk on the cheap path is high. A single well-chosen model is simpler to operate and audit.

## Search terms

llm model routing cost, small model large model routing savings, cascade llm cost calculator, model router cost estimate, tiered llm cost optimization, llm blended cost per request, model escalation retry cost

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json